Skip to content

feat: add app-level indy A/B benches to the performance workflow - #16169

Open
jamesfredley wants to merge 2 commits into
9.0.xfrom
feat/app-bench-ci
Open

feat: add app-level indy A/B benches to the performance workflow#16169
jamesfredley wants to merge 2 commits into
9.0.xfrom
feat/app-bench-ci

Conversation

@jamesfredley

@jamesfredley jamesfredley commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Purpose

Teach CI to measure app-level indy vs classic callsite, not just JMH micros.

This does not change the Grails plugin indy default. That remains #16165.

What landed

  • AppHttpBench helper + unit tests (rejects empty sample sets, clamps fork count)
  • Gated AppBench* specs for latency, app1, and gsp-layout
  • gsp-layout now depends on grails-testing-support-http-client
  • Native Gradle orchestrator: ./gradlew :grails-benchmarks:appIndyBench
  • benchmark.yml app-bench job on the performance label calls that single task (warmup 80 / samples 300 / forks 2)
  • No PowerShell, shell, or Python runner

Local usage

./gradlew :grails-benchmarks:appIndyBench

Optional: -PappBenchWarmup, -PappBenchSamples, -PappBenchForks, -PappBenchOutDir.

Local numbers already posted on #16165

Same harness, same settings: latency 0.98x, app1 0.99x, no clear change. gsp-layout was skipped there because that module lacked the HTTP client dep; this PR adds it.

Test plan

  • AppHttpBenchSpec
  • AppIndyBenchSpec
  • CI performance label runs the new job

Land the gated HTTP harness (latency, app1, gsp-layout) and run it
from benchmark.yml when a PR has the performance label. Compare
-PgrailsIndy=false vs true on HEAD. Does not change the plugin default.

Assisted-by: claude-code:claude-opus-5
Copilot AI lite review requested due to automatic review settings August 18, 2026 21:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the performance-labeled CI workflow to run app-level HTTP benchmarks comparing Groovy indy vs classic callsite dispatch, emitting JMH-compatible JSON so results can be compared with the existing :grails-benchmarks:jmhCompare tooling.

Changes:

  • Add AppHttpBench helper (and unit tests) to measure HTTP request latency and write JMH-shaped JSON output.
  • Add gated AppBench* integration specs in grails-test-examples (latency, app1, gsp-layout) and wire a shared Gradle test configuration to pass bench properties.
  • Add a new app-bench GitHub Actions job (runs noindy then indy, then publishes a comparison report and artifacts) and a local PowerShell runner.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
grails-testing-support-http-client/src/main/groovy/org/apache/grails/testing/http/client/bench/AppHttpBench.groovy New helper to measure request timings and emit JMH-compatible JSON entries.
grails-testing-support-http-client/src/test/groovy/org/apache/grails/testing/http/client/bench/AppHttpBenchSpec.groovy Unit tests for the new bench helper behavior.
grails-test-examples/latency/src/integration-test/groovy/latencyapp/AppBenchFastPingSpec.groovy New gated integration bench for a simple controller render path.
grails-test-examples/latency/build.gradle Applies shared app-bench Gradle wiring to pass bench properties to tests.
grails-test-examples/app1/src/integration-test/groovy/functionaltests/AppBenchInterceptorDemoSpec.groovy New gated integration bench for interceptor + controller path.
grails-test-examples/app1/build.gradle Applies shared app-bench Gradle wiring to pass bench properties to tests.
grails-test-examples/gsp-layout/src/integration-test/groovy/org/example/grails/layout/AppBenchDemoRenderTextSpec.groovy New gated integration bench for GSP layout app request path.
grails-test-examples/gsp-layout/build.gradle Adds HTTP client support dependency for integration tests + applies app-bench wiring.
gradle/app-bench-config.gradle New shared Gradle config to set system properties and disable caching when benches are enabled.
grails-benchmarks/scripts/run-app-indy-bench.ps1 Local runner to execute noindy/indy benches and compare results.
.github/workflows/benchmark.yml Adds app-bench job gated by the performance label and publishes comparison artifacts/summary.
Suppressed comments (2)

grails-benchmarks/scripts/run-app-indy-bench.ps1:96

  • This script is invoked via pwsh (cross-platform), but it calls gradlew.bat, which will fail on non-Windows systems. Use gradlew on Unix-like systems.
    & "$root\gradlew.bat" '-q' ':grails-benchmarks:jmhCompare' `

grails-benchmarks/scripts/run-app-indy-bench.ps1:115

  • This script is invoked via pwsh (cross-platform), but it calls gradlew.bat, which will fail on non-Windows systems. Use gradlew on Unix-like systems.
& "$root\gradlew.bat" '-q' ':grails-benchmarks:jmhCompare' `

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread gradle/app-bench-config.gradle Outdated
Comment thread grails-benchmarks/scripts/run-app-indy-bench.ps1 Outdated
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.07407% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.1404%. Comparing base (cd062bf) to head (b8561d4).

Files with missing lines Patch % Lines
...ails/testing/http/client/bench/AppHttpBench.groovy 74.0741% 20 Missing and 8 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@                Coverage Diff                 @@
##                9.0.x     #16169        +/-   ##
==================================================
+ Coverage     53.1116%   53.1404%   +0.0289%     
- Complexity      19362      19393        +31     
==================================================
  Files            2081       2082         +1     
  Lines           99018      99126       +108     
  Branches        17375      17389        +14     
==================================================
+ Hits            52590      52676        +86     
- Misses          38867      38881        +14     
- Partials         7561       7569         +8     
Files with missing lines Coverage Δ
...ails/testing/http/client/bench/AppHttpBench.groovy 74.0741% <74.0741%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread grails-benchmarks/scripts/run-app-indy-bench.ps1 Outdated
Run app-level indy A/B benches through :grails-benchmarks:appIndyBench.
Nested Gradle is launched via the wrapper JAR, not gradlew or a shell
script. CI now calls that single task. AppHttpBench rejects empty
sample sets and clamps fork count.

Assisted-by: Sisyphus:grok-4.6
@testlens-app

testlens-app Bot commented Aug 19, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: b8561d4
▶️ Tests: 72291 executed
⚪️ Checks: 81/81 completed


Learn more about TestLens at testlens.app/docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants