Skip to content

Fix transient memory amplification in batch_get_documents_tool#4

Merged
mattpodwysocki merged 1 commit intomainfrom
fix-batch-concurrency
Mar 9, 2026
Merged

Fix transient memory amplification in batch_get_documents_tool#4
mattpodwysocki merged 1 commit intomainfrom
fix-batch-concurrency

Conversation

@mattpodwysocki
Copy link
Contributor

@mattpodwysocki mattpodwysocki commented Mar 9, 2026

Summary

The previous cache fix bounded persistent memory growth but left a transient amplification path open: a batch of 20 URLs that all normalize to the same page would still trigger 20 concurrent HTTP fetches and buffer 20 response bodies before the cache could help.

  • Intra-batch URL deduplication: before any fetches are started, URLs are grouped by their normalized key. All inputs that resolve to the same page share a single in-flight Promise — so 20 cache-busting variants of one URL result in exactly 1 HTTP request, not 20.
  • Streaming body size cap (readBodyWithLimit): checks Content-Length upfront and rejects before reading; if Content-Length is absent, aborts the stream as soon as accumulated bytes exceed the 2 MB limit. Avoids buffering oversized responses entirely.
  • Same body size cap applied to get_document_tool.

Test plan

  • 6 new tests in test/tools/batch-get-documents-tool/BatchGetDocumentsTool.test.ts
    • 20 cache-busting URLs → 1 HTTP call, all results return same content
    • 2 distinct normalized URLs → 2 HTTP calls
    • Already-cached URL → 0 HTTP calls
    • Content-Length over limit → per-URL error, batch continues
    • Non-mapbox URLs rejected before any fetch
    • Non-ok HTTP response → per-URL error
  • All 40 existing tests still pass
  • CI green

🤖 Generated with Claude Code

Intra-batch cache-busting URLs (e.g. 20 variants of the same normalized
URL) previously triggered 20 concurrent HTTP fetches and 20 buffered
response bodies before the cache could help.

- Deduplicate by normalized URL within a batch: one in-flight fetch per
  unique normalized key; duplicate inputs share the same Promise
- Add readBodyWithLimit: checks Content-Length before reading and aborts
  the stream if the body exceeds the 2 MB cache entry limit
- Apply the same body size cap to get_document_tool
- Add 6 tests covering deduplication, size limits, and error paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mattpodwysocki mattpodwysocki requested a review from a team as a code owner March 9, 2026 13:56
@mattpodwysocki mattpodwysocki merged commit f62738b into main Mar 9, 2026
1 check passed
@mattpodwysocki mattpodwysocki deleted the fix-batch-concurrency branch March 9, 2026 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants