Skip to content

perf: optimize Java tracing agent (E2E reduction + serialization + writes)#2058

Merged
KRRT7 merged 23 commits intomainfrom
perf/reduce-java-tracer-e2e
Apr 10, 2026
Merged

perf: optimize Java tracing agent (E2E reduction + serialization + writes)#2058
KRRT7 merged 23 commits intomainfrom
perf/reduce-java-tracer-e2e

Conversation

@KRRT7
Copy link
Copy Markdown
Collaborator

@KRRT7 KRRT7 commented Apr 10, 2026 •
edited
Loading

Summary

  • Serialization optimizations: ThreadLocal Kryo Output buffer reuse, inline fast-path for safe arg types, skip verification roundtrip for known-safe containers
  • Write optimizations: Batched SQLite inserts (256/txn), ArrayBlockingQueue, permanent autocommit-off, opt-in in-memory SQLite with VACUUM INTO at shutdown (enabled in CI via is_ci())
  • Profiling fixture: New ProfilingWorkload.java for benchmarking the tracer

Benchmark (50k captures)

Metric Before After Improvement
onEntry ~5200ms ~1200ms 4.3x faster
avg/capture 0.43ms 0.02ms 21x faster
writes ~3200ms ~900ms 3.5x faster
dump to disk N/A ~190ms one-time cost

Test plan

  • 245 Maven unit tests pass
  • 8/8 E2E tests pass (disk-backed mode, local)
  • 8/8 E2E tests pass (in-memory mode, simulating CI)
  • CI passes

KRRT7 added 2 commits April 10, 2026 03:04
@KRRT7
perf: reduce java-tracer E2E from ~75 min to ~15 min
2b0f633 
Remove filterEvens and instanceMethod from the Workload fixture (4→2
functions) and reduce main() loop from 1000→100 rounds. The E2E test
only needs to verify the tracer→optimizer pipeline works end-to-end;
it doesn't need 4 functions or 1604 replay tests to prove that.

Expected impact: ~2 functions × ~8 candidates × fewer replay tests
should bring the job from ~75 min down to ~10-15 min.
@KRRT7
ci: add java_tracer_e2e fixture path to e2e_java change detection
21f61ec 
The fixture directory wasn't in the path filter, so changes to
Workload.java didn't trigger the java E2E tests.
@github-actions github-actions bot added the workflow-modified This PR modifies GitHub Actions workflows label Apr 10, 2026
KRRT7 added 3 commits April 10, 2026 03:17
@KRRT7
fix: update java tracer unit tests for reduced Workload fixture
46957e1 
Remove assertions for filterEvens and instanceMethod which were removed
from the Workload fixture. Adjust expected invocation counts accordingly.
@KRRT7
perf: reduce java-tracer E2E to single function for ~11 min target
08aa94c 
Drop repeatString from the Workload fixture (2→1 function).
computeSum alone exercises the full tracer→optimizer pipeline
(trace → replay tests → optimize → evaluate → rank → explain → review).
The second function added no additional pipeline coverage.
@KRRT7
perf: optimize Java tracing agent serialization and writes
0772398 
- Reuse ThreadLocal Kryo Output buffers (eliminates #1 allocation hotspot)
- Fast-path inline serialization for safe arg types (bypasses executor)
- Skip verification roundtrip for known-safe containers (ArrayList, HashMap, etc.)
- Batch SQLite inserts (256/txn) with permanent autocommit-off
- Switch to ArrayBlockingQueue (no per-element Node allocation)
- Add opt-in in-memory SQLite mode (VACUUM INTO at shutdown), enabled in CI
- Add timing instrumentation (onEntry, serialization, writes, dump)
- Add ProfilingWorkload fixture for benchmarking

Benchmark (50k captures): onEntry 5200ms→1200ms (4.3x), avg/capture
0.43ms→0.02ms (21x), writes 3200ms→900ms (3.5x) with in-memory mode.
@KRRT7 KRRT7 changed the title perf: reduce java-tracer E2E from ~75 min to ~15 min perf: optimize Java tracing agent (E2E reduction + serialization + writes) Apr 10, 2026
KRRT7 added 13 commits April 10, 2026 05:05
@KRRT7
fix: remove stale repeatString assertions from integration tests
e81f25f 
repeatString was removed from Workload.java in the E2E reduction.
@KRRT7
flexing
01e2215
@KRRT7
Remove debug timing instrumentation from tracer
bfe6f3a 
Strip AtomicLong accumulators, System.nanoTime() timing, and
getTimingSummary() that were added for profiling. No functional change.
@KRRT7
fix: drop JFR inline event config that breaks JDK 11
fefccd5 
The jdk.ExecutionSample#period=1ms syntax in -XX:StartFlightRecording
is only supported on JDK 13+. On JDK 11 (CI), it causes
"Failure when starting JFR on_create_vm_2" and no JFR file is created.
The settings=profile preset still provides 10ms CPU sampling.
@KRRT7
chore: add diagnostic logging to compare_test_results
e191f74 
Temporary instrumentation to debug flaky futurehouse E2E test.
Logs matched/skipped/timed-out counts and did_all_timeout state.
@KRRT7
fix: pin PYTHONHASHSEED=0 in test env and enhance diff diagnostics
986654b 
Set PYTHONHASHSEED=0 in test subprocess environments so original and
candidate runs use identical hash behavior, eliminating a source of
non-deterministic return-value comparisons.

Also upgrade diff logging from debug to info level with actual types
and repr values for DID_PASS, RETURN_VALUE, and STDOUT diffs.
@KRRT7
chore: remove diagnostic logging from compare_test_results
82ec301
@KRRT7
fix: ensure language_version is detected before optimization API calls
70260f2 
JavaSupport.ensure_runtime_environment() was never called during the
optimization flow, so _language_version stayed None and the backend
received language_version=null. The LLM had no Java version constraint,
causing it to generate Java 16+ APIs (e.g. Stream.toList()) for Java 11
projects.
@KRRT7
chore: replace console.print with logger.info for Java project detection
b05561e
@KRRT7
perf: use --effort low for java-tracer E2E to reduce CI time
151df77
@KRRT7
perf: reduce Java E2E looping time to 5s and cache runtime JAR build
ecf4e63 
Make TOTAL_LOOPING_TIME configurable via CODEFLASH_LOOPING_TIME env var
(defaults to 10s). Set to 5s in Java E2E CI jobs to cut verification
time per candidate. Also cache the codeflash-runtime JAR keyed on
source hash to skip mvn install when unchanged.
@KRRT7
perf: merge Java tracer into single-pass JVM invocation
0d928f2 
Combine JFR profiling and argument capture agent into one
JAVA_TOOL_OPTIONS string, running the target program once instead of
twice. JFR and javaagent are orthogonal JVM features that coexist
without conflict. Keeps build_jfr_env/build_agent_env for standalone
use.
@KRRT7
fix: drop jdk.ExecutionSample#period from combined JFR opts (unsuppor…
013c83f 
…ted on Java 11)
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 deployed to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Active
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
@KRRT7 KRRT7 had a problem deploying to external-trusted-contributors April 10, 2026 17:55 — with GitHub Actions Error
KRRT7 added 5 commits April 10, 2026 12:58
@KRRT7
fix: skip environment approval gate for trusted users on workflow_dis…
cb87763 
…patch
@KRRT7
ci: add standalone Java E2E workflow for isolated testing
40f16b5
@KRRT7
perf: trim tracer E2E workload to single function (repeatString)
5c778df 
Keep only repeatString which reliably produces 284% improvement.
Drop computeSum (marginal 16%), filterEvens and instanceMethod (no
optimization found). Reduces tracer E2E from ~1h27m to ~21m.
@KRRT7
fix: add --no-pr to codeflash optimize workflow to prevent CI-opened PRs
0cb67c1
@KRRT7
fix: update test assertions to match simplified Workload fixture
b737f71 
The Workload.java fixture was trimmed to only repeatString but test
files still asserted computeSum, filterEvens, and instanceMethod.
@KRRT7 KRRT7 enabled auto-merge April 10, 2026 21:22
@KRRT7 KRRT7 merged commit 819a56c into main Apr 10, 2026
30 of 31 checks passed
@KRRT7 KRRT7 deleted the perf/reduce-java-tracer-e2e branch April 10, 2026 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

workflow-modified This PR modifies GitHub Actions workflows

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@KRRT7