Skip to content

Compute humanized bytes lazily#5

Closed
dnagoda wants to merge 1 commit into
dc.batch-4-reuse-providerfrom
dc.batch-5-lazy-humanized
Closed

Compute humanized bytes lazily#5
dnagoda wants to merge 1 commit into
dc.batch-4-reuse-providerfrom
dc.batch-5-lazy-humanized

Conversation

@dnagoda

@dnagoda dnagoda commented Jul 2, 2026

Copy link
Copy Markdown
Owner

📚 Stack (bottom → top)

  1. Add batch mode for processing multiple inputs via JSONL #1 Add batch mode for processing multiple inputs via JSONL
  2. Preserve JSON input object order #2 Preserve JSON input object order
  3. Refine batch failure handling #3 Refine batch failure handling
  4. Reuse the compiled provider across runs #4 Reuse the compiled provider across runs
  5. Compute humanized bytes lazily #5 Compute humanized bytes lazily
  6. Add minimal batch output with --batch-full-output opt-in #6 Add minimal batch output with --batch-full-output opt-in

Stacked draft PRs on the fork; review bottom-up. Merge target is Shopify/function-runner once upstream write access is available.


Why

BytesContainer eagerly built a humanized String (pretty-printed JSON or a
hex dump) on every construction, even though it's only needed when a
FunctionRunResult is actually displayed. Batch mode constructs a
BytesContainer per input row and never renders the humanized form, so this was
pure per-row overhead.

What

  • Drop the stored humanized field; replace it with a HumanizedBytes Display
    wrapper that formats on demand.
  • Invalid-JSON output now retains its raw bytes so the wrapper can reproduce the
    previous lossy-UTF8 rendering.
  • Single-run output is byte-for-byte unchanged.

Testing

  • cargo build
  • cargo test

BytesContainer eagerly built a `humanized` String (pretty-printed JSON or a
hex dump) on every construction, even though it is only needed when a
FunctionRunResult is displayed. Batch mode constructs a BytesContainer per
input row and never renders the humanized form, so this was pure per-row
overhead.

Remove the stored `humanized` field and replace it with a HumanizedBytes
Display wrapper that formats on demand. Single-run output is unchanged.
Invalid JSON output now retains its raw bytes so the wrapper can reproduce
the previous lossy-UTF8 rendering.

Verified with:
- cargo build
- cargo test

Assisted-By: devx/c659e918-9568-4750-b122-e3890447348a
@dnagoda dnagoda force-pushed the dc.batch-4-reuse-provider branch from 4cd6e69 to a8174bb Compare July 2, 2026 21:51
@dnagoda dnagoda force-pushed the dc.batch-5-lazy-humanized branch from f672955 to 5fea7f9 Compare July 2, 2026 21:51
@dnagoda

dnagoda commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Moved upstream to Shopify#593 now that write access is available. Closing this fork PR.

@dnagoda dnagoda closed this Jul 6, 2026
@dnagoda dnagoda deleted the dc.batch-5-lazy-humanized branch July 6, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant