Skip to content

Commit 910676d

Browse files
dotnet-docker-botdotnet-docker-bot
committed
Update common Docker engineering infrastructure with latest
1 parent 227180b commit 910676d

22 files changed

Lines changed: 519 additions & 127 deletions

‎eng/docker-tools/CHANGELOG.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Docker Tools / ImageBuilder Changelog
2+
3+
All breaking changes and new features in `eng/docker-tools` will be documented in this file.
4+
5+
---
6+
7+
## 2026-03-04: Pre-build validation gated by `preBuildTestScriptPath` variable
8+
9+
The `PreBuildValidation` job condition now checks the new `preBuildTestScriptPath` variable instead of `testScriptPath`.
10+
This allows repos to independently control whether pre-build validation runs, without affecting functional tests.
11+
12+
The new variable defaults to `$(testScriptPath)`, so existing repos that have pre-build tests are not affected.
13+
Repos that do not have pre-build tests can set `preBuildTestScriptPath` to `""` to skip the job entirely.
14+
15+
---
16+
17+
## 2026-02-19: Separate Registry Endpoints from Authentication
18+
19+
- Pull request: [#1945](https://github.com/dotnet/docker-tools/pull/1945)
20+
- Issue: [#1914](https://github.com/dotnet/docker-tools/issues/1914)
21+
22+
Authentication details (`serviceConnection`, `resourceGroup`, `subscription`) have been moved from individual registry endpoints into a centralized `RegistryAuthentication` list.
23+
This fixes an issue where ACR authentication could fail when multiple service connections existed for the same registry.
24+
25+
**Before:** Each registry endpoint embedded its own authentication:
26+
27+
```yaml
28+
publishConfig:
29+
BuildRegistry:
30+
server: $(acr.server)
31+
repoPrefix: "my-prefix/"
32+
resourceGroup: $(resourceGroup)
33+
subscription: $(subscription)
34+
serviceConnection:
35+
name: $(serviceConnectionName)
36+
id: $(serviceConnection.id)
37+
clientId: $(serviceConnection.clientId)
38+
tenantId: $(tenant)
39+
PublishRegistry:
40+
server: $(acr.server)
41+
repoPrefix: "publish/"
42+
resourceGroup: $(resourceGroup)
43+
subscription: $(subscription)
44+
serviceConnection:
45+
name: $(publishServiceConnectionName)
46+
id: $(publishServiceConnection.id)
47+
clientId: $(publishServiceConnection.clientId)
48+
tenantId: $(tenant)
49+
```
50+
51+
**After:** Registry endpoints only contain `server` and `repoPrefix`. Authentication is centralized:
52+
53+
```yaml
54+
publishConfig:
55+
BuildRegistry:
56+
server: $(acr.server)
57+
repoPrefix: "my-prefix/"
58+
PublishRegistry:
59+
server: $(acr.server)
60+
repoPrefix: "publish/"
61+
RegistryAuthentication:
62+
- server: $(acr.server)
63+
resourceGroup: $(resourceGroup)
64+
subscription: $(subscription)
65+
serviceConnection:
66+
name: $(serviceConnectionName)
67+
id: $(serviceConnection.id)
68+
clientId: $(serviceConnection.clientId)
69+
tenantId: $(tenant)
70+
```
71+
72+
How to update:
73+
- Update any publishConfig parameters to match the new structure.
74+
- Multiple registries can share authentication. If two registries use the same ACR server, only one entry is needed in `RegistryAuthentication`.
75+
- The new structure should match [ImageBuilder's Configuration Model](https://github.com/dotnet/docker-tools/tree/a82572386854f15af441c50c6efa698a627e9f2b/src/ImageBuilder/Configuration).
76+
- Update service connection setup (if using `setup-service-connections.yml`):
77+
- The template now supports looking up service connections from `publishConfig.RegistryAuthentication`
78+
- Use the new `usesRegistries` parameter to specify which registries need auth setup:
79+
```yaml
80+
- template: eng/docker-tools/templates/stages/setup-service-connections.yml
81+
parameters:
82+
publishConfig: ${{ variables.publishConfig }}
83+
usesRegistries:
84+
- $(buildRegistry.server)
85+
- $(publishRegistry.server)
86+
```

‎eng/docker-tools/DEV-GUIDE.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -180,14 +180,16 @@ The `stages` variable is a comma-separated string that controls which pipeline s
180180
```yaml
181181
variables:
182182
- name: stages
183-
value: "build,test,publish" # Run all stages
183+
value: "build,test,sign,publish" # Run all stages
184184
```
185185
186186
Common patterns:
187-
- `"build"` - Build only, no tests or publishing
188-
- `"build,test"` - Build and test, but don't publish
187+
- `"build"` - Build only, no tests, signing, or publishing
188+
- `"build,test"` - Build and test, but don't sign or publish
189+
- `"build,test,sign"` - Build, test, and sign, but don't publish
190+
- `"sign"` - Sign only (when re-running failed signing from a previous build)
189191
- `"publish"` - Publish only (when re-running a failed publish from a previous build)
190-
- `"build,test,publish"` - Full pipeline
192+
- `"build,test,sign,publish"` - Full pipeline
191193
192194
**Note:** The `Post_Build` stage is implicitly included whenever `build` is in the stages list. You don't need to specify it separately—it automatically runs after Build to merge image info files and consolidate SBOMs.
193195

@@ -372,11 +374,13 @@ Note: For simple retries of failed jobs, use the Azure Pipelines UI "Re-run fail
372374

373375
| Scenario | stages | sourceBuildPipelineRunId |
374376
|----------|--------|--------------------------|
375-
| Normal full build | `"build,test,publish"` | `$(Build.BuildId)` (default) |
377+
| Normal full build | `"build,test,sign,publish"` | `$(Build.BuildId)` (default) |
376378
| Re-run publish after infra fix | `"publish"` | ID of the successful build run |
377379
| Re-test after infra fix | `"test"` | ID of the build run to test |
380+
| Re-sign after infra fix | `"sign"` | ID of the build run to sign |
378381
| Build only (no publish) | `"build"` | `$(Build.BuildId)` (default) |
379382
| Test + publish (skip build) | `"test,publish"` | ID of the build run |
383+
| Sign + publish (skip build/test) | `"sign,publish"` | ID of the build run |
380384

381385
**In the Azure DevOps UI:**
382386

‎eng/docker-tools/templates/jobs/build-images.yml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,6 @@ jobs:
9191
--architecture $(architecture)
9292
--retry
9393
--digests-out-var 'builtImages'
94-
--acr-subscription '${{ parameters.publishConfig.BuildRegistry.subscription }}'
95-
--acr-resource-group '${{ parameters.publishConfig.BuildRegistry.resourceGroup }}'
9694
$(manifestVariables)
9795
$(imageBuilderBuildArgs)
9896
- template: /eng/docker-tools/templates/steps/publish-artifact.yml@self

‎eng/docker-tools/templates/jobs/post-build.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ parameters:
33
internalProjectName: null
44
publicProjectName: null
55
customInitSteps: []
6+
publishConfig: null
67

78
jobs:
89
- job: Build
@@ -18,6 +19,7 @@ jobs:
1819
parameters:
1920
dockerClientOS: linux
2021
customInitSteps: ${{ parameters.customInitSteps }}
22+
publishConfig: ${{ parameters.publishConfig }}
2123
- template: /eng/docker-tools/templates/steps/download-build-artifact.yml@self
2224
parameters:
2325
targetPath: $(Build.ArtifactStagingDirectory)

‎eng/docker-tools/templates/jobs/publish.yml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,8 +97,6 @@ jobs:
9797
internalProjectName: ${{ parameters.internalProjectName }}
9898
args: >-
9999
copyAcrImages
100-
'${{ parameters.publishConfig.BuildRegistry.subscription }}'
101-
'${{ parameters.publishConfig.BuildRegistry.resourceGroup }}'
102100
'${{ parameters.publishConfig.BuildRegistry.repoPrefix }}'
103101
'${{ parameters.publishConfig.BuildRegistry.server }}'
104102
--os-type '*'

‎eng/docker-tools/templates/jobs/sign-images.yml‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Signs container images using ESRP/Notary v2.
2+
# This job downloads the merged image-info artifact and signs all images listed in it.
3+
parameters:
4+
pool: {}
5+
internalProjectName: null
6+
publicProjectName: null
7+
customInitSteps: []
8+
publishConfig: null
9+
sourceBuildPipelineRunId: ""
10+
11+
jobs:
12+
- job: Sign
13+
pool: ${{ parameters.pool }}
14+
variables:
15+
imageInfoDir: $(Build.ArtifactStagingDirectory)/image-info
16+
steps:
17+
18+
# Install MicroBuild signing plugin for ESRP container image signing
19+
- template: /eng/docker-tools/templates/steps/init-signing-linux.yml@self
20+
parameters:
21+
signType: ${{ parameters.publishConfig.Signing.SignType }}
22+
envFileVariableName: signingEnvFilePath
23+
24+
# Setup docker and ImageBuilder
25+
- template: /eng/docker-tools/templates/steps/init-common.yml@self
26+
parameters:
27+
dockerClientOS: linux
28+
setupImageBuilder: true
29+
customInitSteps: ${{ parameters.customInitSteps }}
30+
publishConfig: ${{ parameters.publishConfig }}
31+
envFilePath: $(signingEnvFilePath)
32+
33+
# Download merged image-info artifact from Post_Build stage (or from a previous pipeline run)
34+
- template: /eng/docker-tools/templates/steps/download-build-artifact.yml@self
35+
parameters:
36+
targetPath: $(imageInfoDir)
37+
artifactName: image-info
38+
pipelineRunId: ${{ parameters.sourceBuildPipelineRunId }}
39+
40+
- template: /eng/docker-tools/templates/steps/run-imagebuilder.yml@self
41+
parameters:
42+
displayName: 🔏 Sign Container Images
43+
internalProjectName: ${{ parameters.internalProjectName }}
44+
args: >-
45+
signImages
46+
$(artifactsPath)/image-info/image-info.json
47+
--registry-override ${{ parameters.publishConfig.BuildRegistry.server }}
48+
--repo-prefix ${{ parameters.publishConfig.BuildRegistry.repoPrefix }}
49+
50+
- template: /eng/docker-tools/templates/steps/run-imagebuilder.yml@self
51+
parameters:
52+
displayName: ✅ Verify Container Image Signatures
53+
internalProjectName: ${{ parameters.internalProjectName }}
54+
args: >-
55+
verifySignatures
56+
$(artifactsPath)/image-info/image-info.json
57+
--registry-override ${{ parameters.publishConfig.BuildRegistry.server }}
58+
--repo-prefix ${{ parameters.publishConfig.BuildRegistry.repoPrefix }}

‎eng/docker-tools/templates/jobs/test-images-linux-client.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717
strategy:
1818
matrix: $[ ${{ parameters.matrix }} ]
1919
${{ if eq(parameters.preBuildValidation, 'true') }}:
20-
condition: and(succeeded(), ne(variables.testScriptPath, ''))
20+
condition: and(succeeded(), ne(variables.preBuildTestScriptPath, ''))
2121
pool: ${{ parameters.pool }}
2222
timeoutInMinutes: ${{ parameters.testJobTimeout }}
2323
steps:

‎eng/docker-tools/templates/stages/build-and-test.yml‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,13 @@ parameters:
33
testMatrixType: platformVersionedOs
44
buildMatrixCustomBuildLegGroupArgs: ""
55
testMatrixCustomBuildLegGroupArgs: ""
6-
customCopyBaseImagesInitSteps: []
7-
customGenerateMatrixInitSteps: []
86
# Custom steps to set up ImageBuilder instead of pulling from MCR (e.g., bootstrap from source).
97
# Runs before ImageBuilder pull. If non-empty, skips the default ImageBuilder pull.
108
customInitSteps: []
9+
# Custom steps that run after ImageBuilder is set up but before copy-base-images runs.
10+
customCopyBaseImagesInitSteps: []
11+
# Custom steps that run after ImageBuilder is set up but before matrix generation runs.
12+
customGenerateMatrixInitSteps: []
1113
# Custom steps that run after ImageBuilder is set up but before the build starts.
1214
# Use for build-specific initialization (e.g., setting variables, additional setup).
1315
customBuildInitSteps: []
@@ -218,6 +220,33 @@ stages:
218220
internalProjectName: ${{ parameters.internalProjectName }}
219221
publicProjectName: ${{ parameters.publicProjectName }}
220222
customInitSteps: ${{ parameters.customInitSteps }}
223+
publishConfig: ${{ parameters.publishConfig }}
224+
225+
################################################################################
226+
# Sign Images
227+
################################################################################
228+
- ${{ if eq(parameters.publishConfig.Signing.Enabled, true) }}:
229+
- stage: Sign
230+
dependsOn: Post_Build
231+
condition: "
232+
and(
233+
ne(stageDependencies.Post_Build.outputs['Build.MergeImageInfoFiles.noImageInfos'], 'true'),
234+
and(
235+
contains(variables['stages'], 'sign'),
236+
or(
237+
and(
238+
succeeded(),
239+
contains(variables['stages'], 'build')),
240+
not(contains(variables['stages'], 'build')))))"
241+
jobs:
242+
- template: /eng/docker-tools/templates/jobs/sign-images.yml@self
243+
parameters:
244+
pool: ${{ parameters.linuxAmd64Pool }}
245+
internalProjectName: ${{ parameters.internalProjectName }}
246+
publicProjectName: ${{ parameters.publicProjectName }}
247+
customInitSteps: ${{ parameters.customInitSteps }}
248+
publishConfig: ${{ parameters.publishConfig }}
249+
sourceBuildPipelineRunId: ${{ parameters.sourceBuildPipelineRunId }}
221250

222251
################################################################################
223252
# Test Images

‎eng/docker-tools/templates/stages/dotnet/publish-config-nonprod.yml‎

Lines changed: 43 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,11 @@ parameters:
3838
type: object
3939
default: {}
4040

41+
# Enable container image signing
42+
- name: enableSigning
43+
type: boolean
44+
default: false
45+
4146

4247
stages:
4348
- template: ${{ parameters.stagesTemplate }}
@@ -53,35 +58,44 @@ stages:
5358
InternalMirrorRegistry:
5459
server: $(acr-staging-test.server)
5560
repoPrefix: $(internalMirrorRepoPrefix)
56-
resourceGroup: $(testResourceGroup)
57-
subscription: $(testSubscription)
58-
serviceConnection:
59-
name: $(internal-mirror-test.serviceConnectionName)
60-
id: $(internal-mirror-test.serviceConnection.id)
61-
clientId: $(internal-mirror-test.serviceConnection.clientId)
62-
tenantId: $(testTenant)
6361

6462
PublicMirrorRegistry:
6563
server: $(public-mirror.server)
6664
repoPrefix: $(publicMirrorRepoPrefix)
67-
resourceGroup: $(public-mirror.resourceGroup)
68-
subscription: $(public-mirror.subscription)
69-
serviceConnection:
70-
name: $(public-mirror.serviceConnectionName)
71-
id: $(public-mirror.serviceConnection.id)
72-
tenantId: $(public-mirror.serviceConnection.tenantId)
73-
clientId: $(public-mirror.serviceConnection.clientId)
7465

7566
BuildRegistry:
7667
server: $(acr-staging-test.server)
77-
resourceGroup: $(testResourceGroup)
78-
subscription: $(testSubscription)
7968
repoPrefix: "${{ parameters.stagingRepoPrefix }}${{ parameters.sourceBuildPipelineRunId }}/"
80-
serviceConnection:
81-
name: $(build-test.serviceConnectionName)
82-
id: $(build-test.serviceConnection.id)
83-
clientId: $(build-test.serviceConnection.clientId)
84-
tenantId: $(testTenant)
69+
70+
PublishRegistry:
71+
server: $(acr-test.server)
72+
repoPrefix: "${{ parameters.publishRepoPrefix }}"
73+
74+
RegistryAuthentication:
75+
- server: $(acr-staging-test.server)
76+
resourceGroup: $(testResourceGroup)
77+
subscription: $(testSubscription)
78+
serviceConnection:
79+
name: $(build-test.serviceConnectionName)
80+
id: $(build-test.serviceConnection.id)
81+
clientId: $(build-test.serviceConnection.clientId)
82+
tenantId: $(testTenant)
83+
- server: $(public-mirror.server)
84+
resourceGroup: $(public-mirror.resourceGroup)
85+
subscription: $(public-mirror.subscription)
86+
serviceConnection:
87+
name: $(public-mirror.serviceConnectionName)
88+
id: $(public-mirror.serviceConnection.id)
89+
tenantId: $(public-mirror.serviceConnection.tenantId)
90+
clientId: $(public-mirror.serviceConnection.clientId)
91+
- server: $(acr-test.server)
92+
resourceGroup: $(testResourceGroup)
93+
subscription: $(testSubscription)
94+
serviceConnection:
95+
name: $(publish-test.serviceConnectionName)
96+
id: $(publish-test.serviceConnection.id)
97+
clientId: $(publish-test.serviceConnection.clientId)
98+
tenantId: $(testTenant)
8599

86100
cleanServiceConnection:
87101
name: $(clean-test.serviceConnectionName)
@@ -95,13 +109,11 @@ stages:
95109
clientId: $(test-nonprod.serviceConnection.clientId)
96110
tenantId: $(testTenant)
97111

98-
PublishRegistry:
99-
server: $(acr-test.server)
100-
resourceGroup: $(testResourceGroup)
101-
subscription: $(testSubscription)
102-
repoPrefix: "${{ parameters.publishRepoPrefix }}"
103-
serviceConnection:
104-
name: $(publish-test.serviceConnectionName)
105-
id: $(publish-test.serviceConnection.id)
106-
clientId: $(publish-test.serviceConnection.clientId)
107-
tenantId: $(testTenant)
112+
Signing:
113+
Enabled: ${{ parameters.enableSigning }}
114+
ImageSigningKeyCode: $(microBuildSigningKeyCode.testing)
115+
ReferrerSigningKeyCode: $(microBuildSigningKeyCode.testing)
116+
# Use signType 'real' even for non-prod to actually sign with the test certificate.
117+
# The 'test' signType skips signing entirely on linux; the test keycode provides a non-production certificate.
118+
SignType: real
119+
TrustStoreName: test

0 commit comments

Comments
 (0)