[release/10.0.1xx] Fix dotnet restore hang during prepare step#11350
Open
jonathanpeppers wants to merge 2 commits into
Open
[release/10.0.1xx] Fix dotnet restore hang during prepare step#11350jonathanpeppers wants to merge 2 commits into
jonathanpeppers wants to merge 2 commits into
Conversation
The `dotnet restore` of `package-download.proj` was hanging during the prepare phase, causing all 3 retry attempts to time out after 10 minutes each with zero stdout output. Two issues fixed: 1. Double-quoting of arguments: `ProcessRunner.QuoteArgument()` was called before passing args to the `ProcessRunner` constructor, but the constructor already quotes all arguments via `AddQuotedArgument()`. This caused paths to be wrapped in double quotes (e.g. `"\"/path/to/file\""`), which could cause `dotnet restore` to hang trying to resolve an invalid path. 2. Added `DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true` to the shared pipeline variables so all CI pipelines skip the .NET first-run experience, which can also hang on CI agents. Fixes: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=14008410 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Increase ProcessTimeout for runtime pack restore to 30 minutes The dotnet restore of package-download.proj downloads ~18 NuGet runtime packs and workload manifests. The default ProcessRunner timeout of 10 minutes is not enough on slower CI machines, causing all 3 retry attempts to time out and the entire Step_InstallDotNetPreview to fail. Increase the ProcessTimeout to 30 minutes for this specific restore. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix dotnet restore hang in Step_InstallDotNetPreview Three changes to address the recurring restore hang on macOS CI: 1. Kill entire process tree on timeout: process.Kill() only kills the top-level dotnet process, leaving orphaned MSBuild worker nodes alive. These can hold locks that block subsequent retry attempts. Use process.Kill(entireProcessTree: true) instead. 2. Replace DOTNET_SKIP_FIRST_TIME_EXPERIENCE (ignored since .NET 5) with DOTNET_NOLOGO and DOTNET_GENERATE_ASPNET_CERTIFICATE=false. The first-run experience still runs ASP.NET cert generation on macOS which involves keychain operations that can hang on CI. 3. Revert the 30-minute ProcessTimeout since 10 minutes is already generous for a restore that should take seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Cherry-picks fixes onto release/10.0.1xx to reduce the likelihood of dotnet restore hanging during the CI prepare phase, improving reliability of workload/runtime pack restoration and timeout cleanup.
Changes:
- Add retry/backoff and more verbose/echoed output when restoring
package-download.projruntime packs inxaprepare. - Ensure timed-out processes are terminated via
Kill(entireProcessTree: true)to prevent orphaned child processes. - Add pipeline variables to suppress first-run banner/side effects (telemetry, logo, ASP.NET cert generation).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| build-tools/xaprepare/xaprepare/Steps/Step_InstallDotNetPreview.cs | Adds retry loop with per-attempt binlogs and stdout/stderr echo for runtime pack restore. |
| build-tools/xaprepare/xaprepare/Application/ProcessRunner.cs | Switches timeout termination to kill the full process tree. |
| build-tools/automation/yaml-templates/variables.yaml | Adds dotnet-related environment variables to reduce first-run friction/hangs on CI. |
jonathanpeppers
added
the
ready-to-review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picks two fixes from
maintorelease/10.0.1xx:dotnet restorehang during prepare step #11282)