Skip to content

Commit 11ae1e9

Browse files
gh-151365: Retry git fetch in CI workflows
Fetching the history for computing the branch diff can fail with "fatal: shallow file has changed since we read it", a git 2.54 bug which is hit especially often for old pull requests, which have a lot of history to fetch. Retry the fetch up to 3 times, and annotate the run with a warning on every retry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 61304a3 commit 11ae1e9

3 files changed

Lines changed: 55 additions & 7 deletions

File tree

.github/workflows/reusable-check-html-ids.yml

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,17 +23,33 @@ jobs:
2323
- name: 'Find merge base'
2424
id: merge-base
2525
run: |
26+
# Fetching can fail with "fatal: shallow file has changed since we
27+
# read it", especially for old pull requests, which have a lot of
28+
# history to fetch. Retry it several times (see gh-151365).
29+
retry_fetch() {
30+
for attempt in $(seq 5); do
31+
if [ "${attempt}" -gt 1 ]; then
32+
echo "::warning::git fetch failed, retrying (attempt ${attempt})"
33+
sleep 3
34+
fi
35+
if git fetch "$@"; then
36+
return 0
37+
fi
38+
done
39+
return 1
40+
}
41+
2642
BASE="${{ github.event.pull_request.base.sha }}"
2743
HEAD="${{ github.event.pull_request.head.sha }}"
28-
git fetch --depth=$((${{ github.event.pull_request.commits }} + 10)) --no-tags origin "$BASE" "$HEAD"
44+
retry_fetch --depth=$((${{ github.event.pull_request.commits }} + 10)) --no-tags origin "$BASE" "$HEAD"
2945
3046
if ! MERGE_BASE=$(git merge-base "$BASE" "$HEAD" 2>/dev/null); then
31-
git fetch --deepen=1 --no-tags origin "$BASE" "$HEAD"
47+
retry_fetch --deepen=1 --no-tags origin "$BASE" "$HEAD"
3248
3349
OLDEST=$(git rev-list --reflog --max-parents=0 --reverse "${BASE}^" "${HEAD}^" | head -1)
3450
TIMESTAMP=$(git show --format=%at --no-patch "$OLDEST")
3551
36-
git fetch --shallow-since="$TIMESTAMP" --no-tags origin "$BASE" "$HEAD"
52+
retry_fetch --shallow-since="$TIMESTAMP" --no-tags origin "$BASE" "$HEAD"
3753
3854
MERGE_BASE=$(git merge-base "$BASE" "$HEAD")
3955
fi

.github/workflows/reusable-context.yml

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,16 +96,32 @@ jobs:
9696
run: |
9797
set -eux
9898
99+
# Fetching can fail with "fatal: shallow file has changed since we
100+
# read it", especially for old pull requests, which have a lot of
101+
# history to fetch. Retry it several times (see gh-151365).
102+
retry_fetch() {
103+
for attempt in $(seq 5); do
104+
if [ "${attempt}" -gt 1 ]; then
105+
echo "::warning::git fetch failed, retrying (attempt ${attempt})"
106+
sleep 3
107+
fi
108+
if git fetch "$@"; then
109+
return 0
110+
fi
111+
done
112+
return 1
113+
}
114+
99115
# Fetch enough history to find a common ancestor commit (aka merge-base):
100-
git fetch origin "${refspec_pr}" --depth=$(( commits + 1 )) \
116+
retry_fetch origin "${refspec_pr}" --depth=$(( commits + 1 )) \
101117
--no-tags --prune --no-recurse-submodules
102118
103119
# This should get the oldest commit in the local fetched history (which may not be the commit the PR branched from):
104120
COMMON_ANCESTOR=$( git rev-list --first-parent --max-parents=0 --max-count=1 "${branch_pr}" )
105121
DATE=$( git log --date=iso8601 --format=%cd "${COMMON_ANCESTOR}" )
106122
107123
# Get all commits since that commit date from the base branch (eg: main):
108-
git fetch origin "${refspec_base}" --shallow-since="${DATE}" \
124+
retry_fetch origin "${refspec_base}" --shallow-since="${DATE}" \
109125
--no-tags --prune --no-recurse-submodules
110126
env:
111127
branch_pr: 'origin/${{ github.event.pull_request.head.ref }}'

.github/workflows/reusable-docs.yml

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,16 +51,32 @@ jobs:
5151
- name: 'Fetch commits to get branch diff'
5252
if: github.event_name == 'pull_request'
5353
run: |
54+
# Fetching can fail with "fatal: shallow file has changed since we
55+
# read it", especially for old pull requests, which have a lot of
56+
# history to fetch. Retry it several times (see gh-151365).
57+
retry_fetch() {
58+
for attempt in $(seq 5); do
59+
if [ "${attempt}" -gt 1 ]; then
60+
echo "::warning::git fetch failed, retrying (attempt ${attempt})"
61+
sleep 3
62+
fi
63+
if git fetch "$@"; then
64+
return 0
65+
fi
66+
done
67+
return 1
68+
}
69+
5470
# Fetch enough history to find a common ancestor commit (aka merge-base):
55-
git fetch origin "${refspec_pr}" --depth=$(( commits + 1 )) \
71+
retry_fetch origin "${refspec_pr}" --depth=$(( commits + 1 )) \
5672
--no-tags --prune --no-recurse-submodules
5773
5874
# This should get the oldest commit in the local fetched history (which may not be the commit the PR branched from):
5975
COMMON_ANCESTOR=$( git rev-list --first-parent --max-parents=0 --max-count=1 "${branch_pr}" )
6076
DATE=$( git log --date=iso8601 --format=%cd "${COMMON_ANCESTOR}" )
6177
6278
# Get all commits since that commit date from the base branch (eg: master or main):
63-
git fetch origin "${refspec_base}" --shallow-since="${DATE}" \
79+
retry_fetch origin "${refspec_base}" --shallow-since="${DATE}" \
6480
--no-tags --prune --no-recurse-submodules
6581
- name: 'Set up Python'
6682
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0

0 commit comments

Comments
 (0)