Stream template build-context upload from disk instead of buffering in memory

[mishushakov]

Summary

Template builds previously buffered the entire gzipped build-context tar archive in memory before uploading it. This PR spools the archive to a temporary file and streams it from disk during upload — in the JS SDK and both sync and async Python SDKs — so memory usage no longer scales with the size of the build context.

The upload keeps an explicit Content-Length header (taken from the spooled file's size), which S3 presigned PUT URLs require — they reject Transfer-Encoding: chunked with 501 NotImplemented (#1243).

Changes

• JS (packages/js-sdk/src/template/): tarFileStream/tarFileStreamUpload are replaced by tarFileToStream, which writes the archive to a temp file and returns a self-cleaning read stream plus its size. The spooled temp file deletes itself once the stream is closed (consumed, errored, or destroyed) via the stream's close event — mirroring the Python SDK's tar_file_stream. buildApi streams this body with duplex: 'half' and an explicit Content-Length from size; if fetch throws before consuming the body, it destroys the stream to trigger the same cleanup. There is no separate cleanup callback, so a cleanup failure can no longer mask the upload result.
• Python (packages/python-sdk/e2b/template/utils.py, template_async/build_api.py, template_sync/build_api.py): tar_file_stream now writes to a tempfile.TemporaryFile instead of io.BytesIO and returns the file object positioned at the start; the upload streams from it with an explicit Content-Length and closes it (deleting the temp file) when done.
• Tests updated for the new return shapes (JS tarFileToStream.test.ts, uploadFile.test.ts; Python upload/tar tests), including assertions that the spooled archive is removed on both the consume and destroy paths.

Usage

No API changes — Template.build() / template builds behave the same, just without holding the build context in memory:

await Template.build(template, { alias: 'my-template' })

Template.build(template, alias="my-template")

Split out of #1433, which covers streaming for sandbox/volume file uploads and downloads.

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: e353629

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
e2b	Patch
@e2b/python-sdk	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cursor]

PR Summary

Medium Risk
Changes core template build upload paths in both SDKs; behavior is intended to be equivalent but relies on streaming fetch/httpx and temp-file lifecycle, with added regression tests for upload headers and cleanup error handling.

Overview
Template build uploads no longer hold the full gzipped build-context tar in RAM. JS and Python (sync and async) now spool the archive to a temp file, upload from disk with an explicit Content-Length (still required for S3 presigned PUTs that reject chunked encoding), and remove the spool when the read stream finishes or is destroyed.

JS: tarFileStream writes to disk and returns { stream, size }; uploadFile uses Readable.toWeb, Content-Length, and duplex: 'half', and destroys the stream on failure so temp dirs are cleaned up. The old in-memory buffer path and tarFileStreamUpload wrapper are gone.

Python: tar_file_stream uses tempfile.TemporaryFile instead of BytesIO; uploads stream the file (async via aiter_io_chunks + Content-Length). Temp-file close/cleanup is best-effort so a cleanup error after a successful upload does not surface as FileUploadException.

Tests cover Content-Length/no-chunked contract, temp cleanup on consume/destroy, and post-upload close failures.

^{Reviewed by Cursor Bugbot for commit e353629. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Package Artifacts

Built from 534aa69. Download artifacts from this workflow run.

JS SDK (e2b@2.30.6-mishushakov-stream-template-build-upload.0):

npm install ./e2b-2.30.6-mishushakov-stream-template-build-upload.0.tgz

CLI (@e2b/cli@2.12.3-mishushakov-stream-template-build-upload.0):

npm install ./e2b-cli-2.12.3-mishushakov-stream-template-build-upload.0.tgz

Python SDK (e2b==2.29.5+mishushakov-stream-template-build-upload):

pip install ./e2b-2.29.5+mishushakov.stream.template.build.upload-py3-none-any.whl

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4c78c2b. Configure here.}