Skip to content

Reuse the compiled provider across runs#4

Closed
dnagoda wants to merge 1 commit into
dc.batch-3-failure-handlingfrom
dc.batch-4-reuse-provider
Closed

Reuse the compiled provider across runs#4
dnagoda wants to merge 1 commit into
dc.batch-3-failure-handlingfrom
dc.batch-4-reuse-provider

Conversation

@dnagoda

@dnagoda dnagoda commented Jul 2, 2026

Copy link
Copy Markdown
Owner

📚 Stack (bottom → top)

  1. Add batch mode for processing multiple inputs via JSONL #1 Add batch mode for processing multiple inputs via JSONL
  2. Preserve JSON input object order #2 Preserve JSON input object order
  3. Refine batch failure handling #3 Refine batch failure handling
  4. Reuse the compiled provider across runs #4 Reuse the compiled provider across runs
  5. Compute humanized bytes lazily #5 Compute humanized bytes lazily
  6. Add minimal batch output with --batch-full-output opt-in #6 Add minimal batch output with --batch-full-output opt-in

Stacked draft PRs on the fork; review bottom-up. Merge target is Shopify/function-runner once upstream write access is available.


Why

Batch mode already reuses the loaded function module, but Javy/provider
functions still recompiled the embedded standard provider for every input
row
. At scale, provider compilation dominated batch runtime.

What

  • Store the compiled provider Module on Provider and compile it once,
    when the ValidatedModule is built.
  • FunctionRunParams now carries a ValidatedModule (built once in main), so
    single and batch runs reuse the already-compiled function and provider
    modules — per-row work is just instantiation.
  • Removes the now-redundant CompiledProvider type, the parallel
    *_with_compiled_provider entry points, and the per-row
    Module::from_binary fallback.
  • Add batch coverage for a Javy v3 function to exercise the provider path.

Performance

Measured locally: 5,000 js_function_javy_plugin_v3 rows complete in
~1.2s, versus ~52s when the provider was recompiled per input row.

Testing

  • cargo test
  • cargo build --release

Batch mode reuses the loaded function module, but Javy/provider functions
still recompiled the embedded standard provider for every input row, so
provider setup dominated batch runtime.

Store the compiled provider Module on Provider and compile it once when the
ValidatedModule is built. FunctionRunParams now carries a ValidatedModule
(built once in main), so single and batch runs reuse the already-compiled
function and provider modules and per-row work is just instantiation. This
drops the separate CompiledProvider type, the parallel
*_with_compiled_provider entry points, and the per-row Module::from_binary
fallback.

Add batch coverage for a Javy v3 function to exercise the provider path.

Measured locally: 5,000 js_function_javy_plugin_v3 rows complete in ~1.2s,
versus ~52s when the provider was recompiled per input row.

Verified with:
- cargo test
- cargo build --release

Assisted-By: devx/c659e918-9568-4750-b122-e3890447348a
@dnagoda dnagoda force-pushed the dc.batch-3-failure-handling branch from 0ecdade to 388e7a4 Compare July 2, 2026 21:51
@dnagoda dnagoda force-pushed the dc.batch-4-reuse-provider branch from 4cd6e69 to a8174bb Compare July 2, 2026 21:51
@dnagoda

dnagoda commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Moved upstream to Shopify#592 now that write access is available. Closing this fork PR.

@dnagoda dnagoda closed this Jul 6, 2026
@dnagoda dnagoda deleted the dc.batch-4-reuse-provider branch July 6, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant