feat: add app-level indy A/B benches to the performance workflow - #16169
feat: add app-level indy A/B benches to the performance workflow#16169jamesfredley wants to merge 2 commits into
Conversation
Land the gated HTTP harness (latency, app1, gsp-layout) and run it from benchmark.yml when a PR has the performance label. Compare -PgrailsIndy=false vs true on HEAD. Does not change the plugin default. Assisted-by: claude-code:claude-opus-5
There was a problem hiding this comment.
Pull request overview
This PR extends the performance-labeled CI workflow to run app-level HTTP benchmarks comparing Groovy indy vs classic callsite dispatch, emitting JMH-compatible JSON so results can be compared with the existing :grails-benchmarks:jmhCompare tooling.
Changes:
- Add
AppHttpBenchhelper (and unit tests) to measure HTTP request latency and write JMH-shaped JSON output. - Add gated
AppBench*integration specs ingrails-test-examples(latency, app1, gsp-layout) and wire a shared Gradle test configuration to pass bench properties. - Add a new
app-benchGitHub Actions job (runs noindy then indy, then publishes a comparison report and artifacts) and a local PowerShell runner.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| grails-testing-support-http-client/src/main/groovy/org/apache/grails/testing/http/client/bench/AppHttpBench.groovy | New helper to measure request timings and emit JMH-compatible JSON entries. |
| grails-testing-support-http-client/src/test/groovy/org/apache/grails/testing/http/client/bench/AppHttpBenchSpec.groovy | Unit tests for the new bench helper behavior. |
| grails-test-examples/latency/src/integration-test/groovy/latencyapp/AppBenchFastPingSpec.groovy | New gated integration bench for a simple controller render path. |
| grails-test-examples/latency/build.gradle | Applies shared app-bench Gradle wiring to pass bench properties to tests. |
| grails-test-examples/app1/src/integration-test/groovy/functionaltests/AppBenchInterceptorDemoSpec.groovy | New gated integration bench for interceptor + controller path. |
| grails-test-examples/app1/build.gradle | Applies shared app-bench Gradle wiring to pass bench properties to tests. |
| grails-test-examples/gsp-layout/src/integration-test/groovy/org/example/grails/layout/AppBenchDemoRenderTextSpec.groovy | New gated integration bench for GSP layout app request path. |
| grails-test-examples/gsp-layout/build.gradle | Adds HTTP client support dependency for integration tests + applies app-bench wiring. |
| gradle/app-bench-config.gradle | New shared Gradle config to set system properties and disable caching when benches are enabled. |
| grails-benchmarks/scripts/run-app-indy-bench.ps1 | Local runner to execute noindy/indy benches and compare results. |
| .github/workflows/benchmark.yml | Adds app-bench job gated by the performance label and publishes comparison artifacts/summary. |
Suppressed comments (2)
grails-benchmarks/scripts/run-app-indy-bench.ps1:96
- This script is invoked via
pwsh(cross-platform), but it callsgradlew.bat, which will fail on non-Windows systems. Usegradlewon Unix-like systems.
& "$root\gradlew.bat" '-q' ':grails-benchmarks:jmhCompare' `
grails-benchmarks/scripts/run-app-indy-bench.ps1:115
- This script is invoked via
pwsh(cross-platform), but it callsgradlew.bat, which will fail on non-Windows systems. Usegradlewon Unix-like systems.
& "$root\gradlew.bat" '-q' ':grails-benchmarks:jmhCompare' `
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 9.0.x #16169 +/- ##
==================================================
+ Coverage 53.1116% 53.1404% +0.0289%
- Complexity 19362 19393 +31
==================================================
Files 2081 2082 +1
Lines 99018 99126 +108
Branches 17375 17389 +14
==================================================
+ Hits 52590 52676 +86
- Misses 38867 38881 +14
- Partials 7561 7569 +8
🚀 New features to boost your workflow:
|
Run app-level indy A/B benches through :grails-benchmarks:appIndyBench. Nested Gradle is launched via the wrapper JAR, not gradlew or a shell script. CI now calls that single task. AppHttpBench rejects empty sample sets and clamps fork count. Assisted-by: Sisyphus:grok-4.6
✅ All tests passed ✅🏷️ Commit: b8561d4 Learn more about TestLens at testlens.app/docs. |
Purpose
Teach CI to measure app-level indy vs classic callsite, not just JMH micros.
This does not change the Grails plugin indy default. That remains #16165.
What landed
AppHttpBenchhelper + unit tests (rejects empty sample sets, clamps fork count)AppBench*specs for latency, app1, and gsp-layoutgsp-layoutnow depends ongrails-testing-support-http-client./gradlew :grails-benchmarks:appIndyBenchbenchmark.ymlapp-benchjob on theperformancelabel calls that single task (warmup 80 / samples 300 / forks 2)Local usage
Optional:
-PappBenchWarmup,-PappBenchSamples,-PappBenchForks,-PappBenchOutDir.Local numbers already posted on #16165
Same harness, same settings: latency 0.98x, app1 0.99x, no clear change. gsp-layout was skipped there because that module lacked the HTTP client dep; this PR adds it.
Test plan
AppHttpBenchSpecAppIndyBenchSpecperformancelabel runs the new job