Consolidate single-consumer library test legs into build jobs#128095
Open
agocke wants to merge 2 commits into
Open
Consolidate single-consumer library test legs into build jobs#128095agocke wants to merge 2 commits into
agocke wants to merge 2 commits into
Conversation
Merge 6 split build+test leg pairs into self-contained jobs that build and send to Helix directly, eliminating artifact upload/download overhead. Consolidated legs (Libraries_CheckedCoreCLR): - linux_musl_x64: build+test merged, removed from separate test run - windows_x86: build+test merged, removed from separate test run Consolidated legs (CoreCLR_ReleaseLibraries): - linux_x64: build+test merged, removed separate test run block - linux_musl_arm: build+test merged, removed separate test run block - linux_musl_arm64: build+test merged, removed separate test run block - windows_x64: build+test merged, removed separate test run block Each of these artifacts was only consumed by a single downstream test leg on the same platform. The consolidation follows the existing self-contained pattern used by NativeAOT_Libraries, Mono_Interpreter, and Mono_MiniJIT legs. Net result: -62 lines of pipeline YAML, 6 fewer pipeline jobs, 6 fewer artifact upload+download round-trips. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
Tagging subscribers to this area: @dotnet/area-infrastructure-libraries |
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Consolidates several single-consumer library build→artifact→test legs into self-contained build jobs that submit to Helix directly, reducing pipeline job count and artifact transfer overhead.
Changes:
- Replaced artifact upload steps with direct Helix submission in consolidated build jobs.
- Added
helixQueuesTemplate/helixQueueGroup: librariesto the affected platform matrices. - Increased job timeouts to account for combined build + Helix submission time and removed now-redundant downstream test-run blocks/platforms.
| archiveExtension: $(archiveExtension) | ||
| tarCompression: $(tarCompression) | ||
| artifactName: Libraries_CheckedCoreCLR_TestArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig) | ||
| creator: dotnet-bot |
| artifactName: CoreCLR_ReleaseLibraries_TestArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig) | ||
| creator: dotnet-bot | ||
| buildConfig: Release | ||
| testRunNamePrefixSuffix: coreclr_Checked |
| archiveExtension: $(archiveExtension) | ||
| tarCompression: $(tarCompression) | ||
| artifactName: Libraries_CheckedCoreCLR_TestArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig) | ||
| creator: dotnet-bot |
| archiveExtension: $(archiveExtension) | ||
| tarCompression: $(tarCompression) | ||
| artifactName: CoreCLR_ReleaseLibraries_TestArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig) | ||
| creator: dotnet-bot |
The global-build-job.yml template forwards buildConfig to postBuildStep templates automatically (line 253). Passing buildConfig again in the helix.yml parameters block caused a duplicate key error. Fix: set buildConfig to Release at the platform-matrix level (matching the libraries configuration) and hardcode -rc Checked in buildArgs for the CLR runtime configuration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was generated with the assistance of GitHub Copilot.
Summary
Merges 6 split build→upload→download→test leg pairs into self-contained jobs that build and send to Helix directly, eliminating artifact upload/download overhead.
Each of these artifacts was only consumed by a single downstream test leg on the same platform, making them safe consolidation candidates. The change follows the existing self-contained pattern used by NativeAOT_Libraries, Mono_Interpreter, and Mono_MiniJIT legs.
Consolidated legs
Libraries_CheckedCoreCLR → Libraries Test Run
CoreCLR_ReleaseLibraries → Libraries Test Run
What changed
upload-artifact-step.ymlcalls withhelix.ymlinpostBuildStepshelixQueuesTemplate+helixQueueGroup: librariesto platform-matrixbuildConfig: Releaseexplicitly overridden for CoreCLR_ReleaseLibraries (build usescheckedfor CLR, but Helix needsReleasefor libraries config)CoreCLR_ReleaseLibrariestest run block (all 4 platforms consolidated)windows_x86andlinux_musl_x64fromLibraries_CheckedCoreCLRtest run block (keptlinux_arm64which is multi-consumer)Not changed (multi-consumer, left for future work)
CoreCLR_Librariesfor linux_x64, windows_x64, osx_arm64, windows_x86 (consumed by both Installer and Libraries Test Run)Libraries_CheckedCoreCLRfor linux_arm64, osx_arm64 (consumed by both Runtime Tests Run and Libraries Test Run)Net result