cache: redesign as a Bazel-style CAS and action cache - #4539
Merged
Conversation
Replace the ad-hoc on-disk caches with a design modeled on Bazel's local disk cache: - A content-addressable store (cas/) holding blobs keyed by the BLAKE3 hash of their contents, written atomically and verified on read, with corrupt entries evicted and repaired. - An action cache (ac/) mapping the digest of a unit of cacheable work and its inputs to the CAS digests of its outputs. Entries whose outputs are missing from the CAS are treated as misses. The query analysis cache becomes a QueryAnalysis action keyed by the sqlc version, config, schema, and query, with the marshaled analysis stored in the CAS. The WASM plugin cache becomes a FetchPlugin action keyed by the URL and expected sha256, with the plugin binary stored in the CAS; the declared sha256 is still re-verified on every cache hit. wazero's compiled-code cache keeps its own directory (wazero/) since its format is managed by the runtime. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
A remote fetch with a declared sha256 doesn't need the action cache: the checksum is itself a content address. Make the CAS hash-function aware — blobs live at cas/<function>/<xx>/<hash>, keyed by BLAKE3 by default — and add PutSHA256/SHA256Digest so remotely fetched content is stored and looked up directly under the checksum declared in the configuration, with the checksum re-verified on every read. The WASM plugin path drops its FetchPlugin action entirely; the action cache now backs only work whose outputs aren't derivable from its inputs, like query analysis. This mirrors Bazel, where downloads go through the checksum-addressed repository cache rather than the action cache. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
The cache doesn't hold anything big enough for hash speed to matter, so drop BLAKE3 (and the lukechampine.com/blake3 dependency) and key everything — CAS blobs, action digests — by SHA-256. The CAS layout flattens back to cas/<xx>/<hash>, and a remotely fetched plugin's address is now literally the checksum declared in the configuration file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
Compiling a module to machine code is cacheable work like any other, so model it as a CompileModule action keyed by the module checksum, wazero version, GOOS, and GOARCH instead of handing wazero a private directory outside the cache's discipline. The action cache gains tree-shaped outputs for tools that read and write output directories: PutTree stores every file under a directory as a named output blob in the CAS, and GetTree materializes them back. Compiled machine code is materialized into exec/<action-hash>, which wazero's compilation cache reads on start; like the rest of the cache, exec trees are safe to delete because the authoritative bytes live in the CAS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
Version strings under-invalidate: every dev build reports the same version, so a rebuilt sqlc with different analysis or codegen logic would keep hitting entries produced by the old binary. Hash the running executable once per process and mix it into every action digest as the first input, the way Bazel treats the toolchain as action input. This subsumes the explicit version, wazero version, and GOOS/GOARCH inputs, which are all determined by the binary, so drop them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
Hold an os.Root for the cache directory instead of a path string in CAS and ActionCache. Every entry read, write, rename, and eviction now goes through the root, so no entry name — hash-derived or deserialized from an action cache entry — can traverse outside the cache directory. Cache handles now hold a directory fd, so Open gains a matching Close: the analyzer opens the cache once per instance and closes it with the analyzer, and the WASM runner closes it after loading a plugin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
Hashing the ~100MB sqlc binary costs ~80ms, which every short-lived sqlc process paid on its first action. Memoize the digest on disk keyed by the executable's path, size, and mtime — the same trick as Bazel's file digest cache — so a warm run costs a stat and a tiny read, and only a rebuilt or moved binary is re-hashed. This puts warm-cache generate times back on par with the pre-redesign cache (~32ms for a WASM plugin run, previously ~100ms with per-process hashing). The memo lives in the cache, so NewAction moves onto Cache. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
Address three issues in the action cache's tree materialization, found in adversarial review: 1. GetTree wrote output blobs to filepath.Join(dir, rel) with plain os calls, where rel is an output name read from the (non-self-validating) action cache entry. A tampered entry with a "../" name could escape the exec directory and write attacker-controlled CAS content anywhere the process could — contradicting the confinement the rest of the cache gets from os.Root. Reject any rel that is not filepath.IsLocal. 2. GetTree trusted an already-present file by size alone and never re-hashed it, and staged writes without fsync. A right-sized but torn or corrupt exec file (power loss, bitrot) was therefore trusted forever: wazero hard-fails deserializing it instead of recompiling, and the size match kept GetTree from ever repairing it from the CAS, bricking codegen until a manual cache wipe. Reuse an existing file only when it still hashes to the digest, and stage writes through an fsynced temp + rename. 3. The exec directory was shared across processes at exec/<action-hash>. wazero stages <key>.tmp files in place while compiling, which a concurrent process's PutTree WalkDir could sweep into its action result. ExecDir now returns a fresh private directory per call, and the WASM runner removes it once wazero has loaded the module; the compiled bytes remain reproducible from the CAS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Redesigns sqlc's on-disk cache after Bazel's local disk cache: a content-addressable store keyed by SHA-256 plus an action cache, replacing the ad-hoc query analysis cache and WASM plugin cache.
Design
The cache root (
SQLCCACHE, defaulting to the user cache dir) now holds:cas/<xx>/<hash>— content-addressable blobs keyed by the SHA-256 of their contents: plugin binaries, query analysis protos, compiled WASM machine code. Writes are atomic (temp + rename), reads re-hash and evict corrupt entries, andPutrepairs corrupt entries in place, so concurrent sqlc processes race benignly.ac/<xx>/<hash>— action cache entries mapping the digest of a unit of work (a mnemonic plus length-prefix-framed inputs) to the CAS digests of its outputs. A hit is only reported if every referenced output still exists in the CAS, like Bazel's disk cache.exec/<action-hash>/— materialized output trees for tools that read outputs from disk (wazero's compilation cache). Safe to delete; the authoritative bytes live in the CAS.The sha256 of the sqlc binary itself is an implicit first input of every action, the way Bazel treats the toolchain as an action input: version strings under-invalidate (every dev build reports the same version), while the binary hash guarantees a rebuilt sqlc never reuses entries produced by different analysis or codegen logic. It also subsumes the embedded wazero version and GOOS/GOARCH as compile-action inputs.
All storage I/O is confined to the cache directory through an
os.Root, so no entry name — hash-derived or deserialized from an action cache entry — can traverse outside it.What uses it
QueryAnalysisaction keyed by config + schema + query, with the analysis proto stored in the CAS. Replaces the FNV-64-keyed flat-file cache.CompileModuleaction keyed by the module checksum (plus the implicit binary hash). wazero has no public API to serialize compiled modules, so on a miss it compiles into the action'sexec/dir and the resulting files are lifted into the CAS (PutTree); on a hit they're materialized back (GetTree) for wazero's file cache to load instead of recompiling. Replaces the previous long-lived wazero-managed directory.Testing
--tags=examplessuite passes against live PostgreSQL and MySQL, including all WASM plugin end-to-end tests.sqlc-gen-greeter: first run populates the cache, a fully offline run generates from cache alone, a tampered blob is detected, evicted, and re-fetched, and after deletingexec/the machine code is rematerialized byte-identical from the CAS.CompileModuleaction entries — which both point at the same deduplicated machine-code blob in the CAS, since the embedded wazero is identical.🤖 Generated with Claude Code
https://claude.ai/code/session_01MPgq3rwip76D554BktbqR4