Skip to content

feat: Add incremental git pull-through #401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nsextonTT wants to merge 12 commits into
base: main
Choose a base branch
from
Open

feat: Add incremental git pull-through #401

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
0d080c9
gitclone: add batch missing-object detection
nsextonTT Jul 31, 2026
625473c
git: parse upload-pack request bodies
nsextonTT Jul 31, 2026
99ac935
gitclone: report whether a coalescing fetch ran
nsextonTT Jul 31, 2026
ba951be
gitclone: singleflight the upstream ref check
nsextonTT Jul 31, 2026
25018d0
git: add incremental pull-through metrics
nsextonTT Jul 31, 2026
5af6d6f
git: serve partially-cached fetches from the mirror
nsextonTT Aug 2, 2026
0652ed3
git: cap buffered upload-pack request bodies
nsextonTT Aug 2, 2026
0706bf7
docs: describe incremental pull-through
nsextonTT Jul 31, 2026
85bbff6
git: skip network-dependent integration tests in CI
nsextonTT Jul 31, 2026
8ff8d79
hack: add incremental pull-through report
nsextonTT Jul 31, 2026
a675584
Return ref-check errors from checkRefsStale to the caller
nsextonTT Aug 5, 2026
7ecfff7
fix(git): wait for in-flight fetches before honoring the cooldown
nsextonTT Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ Cachew (pronounced "cashew") is a tiered, protocol-aware, caching HTTP proxy for

### Git

Caches Git repositories with two complementary techniques:
Caches Git repositories with three complementary techniques:

1. **Snapshots** — periodic `.tar.zst` archives that restore 4–5x faster than `git clone`.
2. **Pack caching** — passthrough caching of packs from `git-upload-pack` for incremental pulls.
3. **Incremental pull-through**: when the mirror lacks a wanted object, fetches only the missing delta from upstream and serves the pack locally instead of forwarding the whole request. On by default; set `incremental-pullthrough = false` to disable.

Redirect Git traffic through cachew:

Expand All @@ -26,8 +27,10 @@ cachew git restore https://github.com/org/repo ./repo

```hcl
git {
snapshot-interval = "1h"
repack-interval = "1h"
snapshot-interval = "1h"
repack-interval = "1h"
incremental-pullthrough = true # default; false disables incremental pull-through
incremental-fetch-timeout = "2m" # default
}
```

Expand Down
427 changes: 427 additions & 0 deletions hack/cachew-incremental-report.sh
View file
Open in desktop

Large diffs are not rendered by default.

Loading