ci: retry the stack image pull on a transient registry error - #4
Merged
Conversation
A pull of quay.io/minio/minio dropped one 41 MiB layer partway through, and the daemon's own retry budget is per layer: once those five attempts were spent the whole pull failed, taking the images that had already arrived with it. The spooling leg is the one that reaches quay.io, which is why that leg is where this shows up. Wrap the pull in three attempts with a widening pause. Layers that did arrive stay in the local content store, so a second attempt refetches only what is missing and costs seconds rather than a rerun of the job. A rate limit and a name the registry does not serve are reported as themselves without retrying. Neither improves by asking again, and the explicit pull exists so that those two are legible in the step list rather than surfacing as a startup timeout several minutes later. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
adwk67
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
spoolingleg failed ondocker compose pullwithunexpected EOF:one 41 MB layer of
quay.io/minio/miniodropped partway through. Docker'sown retry budget is per layer and defaults to five attempts, and once
those were spent the whole pull failed, discarding the three images that
had already downloaded. The step lasted 57s, so the job died on about a
minute of bad luck. Memory was 14 GiB free and no OOM was recorded.
Only the spooling leg reaches quay.io, which is why it shows up there:
postgres:17,trinodb/trino:483)minio,mc)This wraps the pull in three attempts with a widening pause. Layers that
did arrive stay in the local content store, so a second attempt refetches
only what is missing and costs seconds instead of a manual job rerun.
A rate limit (
toomanyrequests) and a name the registry does not serve(
manifest unknown,unauthorized,denied): exit immediately with anamed error for inspection.