Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions docs/operations/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,10 +230,12 @@ The wrappers above benchmark whatever is currently checked out. To answer "is
branch B faster or slower than branch A?" fairly on a noisy laptop, use the
dedicated A/B scripts. They **interleave** the two branches (A,B,A,B,…) so
thermal drift averages out, **repeat** each branch and compare **medians**, and
**cool down** between runs. Each branch is rebuilt (`install -pl :graph-compose-core`) before its
runs so the benchmark measures that branch's engine, and untracked benchmark
probes are moved aside around the branch switch so they cannot break the other
branch's compile.
**cool down** between runs. Each branch is rebuilt before its runs so the benchmark measures that branch:
the engine, plus `render-pdf` and `templates`, because the benchmarks module
resolves all three at the branch's own version. Installing only the engine would
leave both sides sharing whichever backend and template jars happened to be in
`~/.m2`. Untracked benchmark probes are moved aside around the branch switch so
they cannot break the other branch's compile.

- **Windows (PowerShell)** — `scripts/ab-bench.ps1`, full suite (latency,
throughput, scalability, stress, comparative):
Expand Down
21 changes: 21 additions & 0 deletions scripts/ab-bench.sh
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,27 @@ build_engine_and_classpath() {
local engine_pl="."
[ -f core/pom.xml ] && engine_pl=":graph-compose-core"
"$MVNW" -B -ntp -DskipTests install -pl "$engine_pl" >/dev/null
# The benchmark also resolves render-pdf and templates (plus the templates
# tests-classifier jar, which is never published) at the branch's own version,
# so those have to be rebuilt per branch as well. Install only the engine and
# both sides resolve the SAME jars from ~/.m2 whenever the two branches share a
# version — the normal case for two branches off develop — so every change in
# the PDF backend or the templates module reports a delta of exactly zero.
# Guarded by existence: a pre-split layout builds them from the root instead.
# render-pdf contributes only its main jar, so skip building its tests too:
# -DskipTests still compiles them, and a branch whose render-pdf tests do not
# compile would abort the whole A/B at the install step with its output sent to
# /dev/null. templates must keep -DskipTests, because the benchmark resolves its
# tests-classifier jar and maven.test.skip would not produce one.
local module
for module in render-pdf templates; do
[ -f "$module/pom.xml" ] || continue
if [ "$module" = "render-pdf" ]; then
"$MVNW" -B -ntp -f "$module/pom.xml" -Dmaven.test.skip=true install >/dev/null
else
"$MVNW" -B -ntp -f "$module/pom.xml" -DskipTests install >/dev/null
fi
done
"$MVNW" -B -ntp -f benchmarks/pom.xml test-compile dependency:build-classpath \
-DincludeScope=test -Dmdep.outputFile=target/benchmark.classpath >/dev/null
CP="benchmarks/target/test-classes${SEP}benchmarks/target/classes${SEP}$(cat benchmarks/target/benchmark.classpath)"
Expand Down
16 changes: 16 additions & 0 deletions scripts/run-benchmarks.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,22 @@ try {
& $mavenWrapper "-B" "-ntp" "-DskipTests" "install" "-pl" $enginePl
}

# The benchmark also resolves render-pdf and templates (plus the templates
# tests-classifier jar, which is never published) at the branch's own
# version. Installing only the engine leaves those coming from ~/.m2, so an
# A/B between two branches that share a version compares the SAME backend and
# template jars on both sides and reports a delta of exactly zero for any
# change in them. Guarded by existence: a pre-split layout builds them from
# the root instead.
foreach ($module in @('render-pdf', 'templates')) {
$modulePom = Join-Path $repoRoot "$module/pom.xml"
if (Test-Path $modulePom) {
Invoke-LoggedCommand -Name "01-install-$module" -Command {
& $mavenWrapper "-B" "-ntp" "-f" $modulePom "-DskipTests" "install"
}
}
}

Invoke-LoggedCommand -Name "01-build-classpath" -Command {
& $mavenWrapper "-B" "-ntp" "-f" $benchmarksPom "test-compile" "dependency:build-classpath" "-DincludeScope=test" "-Dmdep.outputFile=target/benchmark.classpath"
}
Expand Down