ci(vcpkg): Switch binary cache from local files to GitHub Packages NuGet feed#2542
ci(vcpkg): Switch binary cache from local files to GitHub Packages NuGet feed#2542bobtista wants to merge 1 commit intoTheSuperHackers:mainfrom
Conversation
|
| Filename | Overview |
|---|---|
| .github/workflows/build-toolchain.yml | Replaces files-based vcpkg binary cache with GitHub Packages NuGet feed; adds packages:write permission, NuGet auth step via vcpkg-bundled nuget binary, and removes old cache key/restore/save steps — minor idempotency concern on nuget sources add |
| .github/workflows/ci.yml | Adds packages:write permission to the caller workflow so the GITHUB_TOKEN can write to GitHub Packages when delegated through the reusable build-toolchain.yml |
Sequence Diagram
sequenceDiagram
participant GHA as GitHub Actions
participant vRun as lukka/run-vcpkg
participant vcpkg as vcpkg.exe
participant NuGet as NuGet CLI
participant GPkg as GitHub Packages
GHA->>vRun: Setup vcpkg (doNotCache:true)
vRun-->>GHA: VCPKG_ROOT set
GHA->>vcpkg: fetch nuget
vcpkg-->>GHA: path/to/nuget.exe
GHA->>NuGet: sources add GitHubPackages (GITHUB_TOKEN)
NuGet-->>GHA: source registered
GHA->>NuGet: setapikey (GITHUB_TOKEN)
NuGet-->>GHA: API key stored
GHA->>vcpkg: cmake --preset (triggers vcpkg install)
vcpkg->>GPkg: GET package@{abi-hash}
alt cache hit
GPkg-->>vcpkg: package restored (e.g. 28.8ms for ffmpeg)
else cache miss
GPkg-->>vcpkg: 404 Not Found
vcpkg->>vcpkg: build from source
vcpkg->>GPkg: PUT package@{abi-hash}
end
Prompt To Fix All With AI
This is a comment left during a code review.
Path: .github/workflows/build-toolchain.yml
Line: 120-127
Comment:
**`nuget sources add` is not idempotent**
If this step is retried (e.g., via GitHub Actions "Re-run failed jobs") or runs on a self-hosted runner where the `GitHubPackages` source was previously registered, `nuget sources add` will exit with "Source already exists" and fail the step. For ephemeral GitHub-hosted runners this is fine, but wrapping it in an existence check would make the step resilient to retries:
```
$src = "https://nuget.pkg.github.com/${{ github.repository_owner }}/index.json"
$exists = (& $nuget sources list) -match "GitHubPackages"
if ($exists) {
& $nuget sources update -Name GitHubPackages -Source $src `
-StorePasswordInClearText `
-UserName "${{ github.repository_owner }}" `
-Password "${{ secrets.GITHUB_TOKEN }}"
} else {
& $nuget sources add -Name GitHubPackages -Source $src `
-StorePasswordInClearText `
-UserName "${{ github.repository_owner }}" `
-Password "${{ secrets.GITHUB_TOKEN }}"
}
& $nuget setapikey "${{ secrets.GITHUB_TOKEN }}" -Source $src
```
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "ci(vcpkg): Switch binary cache from loca..." | Re-trigger Greptile
We use ephemeral GitHub-hosted runners, do we need to worry about this? Every job starts with a clean NuGet config |
1 participant
Follow up for #2371 and #2515.
Replaces the local file-based vcpkg binary cache (
actions/cache+filesbackend) with a NuGet feed hosted on GitHub Packages. This is a structural change to how vcpkg caching works in CI.Problem
The previous approach used
actions/cachewith vcpkg'sfilesbinary source. This required our GHA cache key to perfectly predict vcpkg's ABI hash output — two separate systems we were trying to keep in sync by hand. Any discrepancy (compiler updates on the runner, host triplet ABI drift, toolchain changes) caused a doom loop: the GHA cache key matched an old entry, vcpkg found zero usable binaries inside it, rebuilt everything (~17 min for ffmpeg), and the save step was skipped becausecache-hitwastrue. PRs #1862, #1973, #2028, #2371, and #2515 each fixed a specific trigger but left the structural vulnerability intact.Solution
With NuGet binary caching, vcpkg uses its own ABI hashes directly as package identifiers in the feed. If a compiler update changes an ABI hash, vcpkg asks the feed for a package with the new hash, doesn't find it, builds once, uploads it, and every subsequent run restores it. There is no GHA cache key to get wrong, no save gate, and no doom loop possible. Based on the same pattern being used in the canonical/multipass repo
Changes
VCPKG_BINARY_SOURCESfromfilesbackend tonugetbackend pointing at GitHub PackagesVCPKG_NUGET_REPOSITORYfor GitHub Packages package-repository associationGITHUB_TOKENpackages: writepermission to bothci.ymlandbuild-toolchain.ymlactions/cache/restore,actions/cache/save, local cache directory setupTest plan