From 6f2873fae9c9d10637cde94a5b9694e09dc12fe8 Mon Sep 17 00:00:00 2001 From: DemchaAV Date: Wed, 3 Jun 2026 09:55:13 +0100 Subject: [PATCH] feat(testing): visual-regression public docs + dogfood + APPROVE_PROPERTY (Track N N2-N4, @since 1.6.9) Completes the public visual-regression surface introduced in #126: - expose PdfVisualRegression.APPROVE_PROPERTY (@since 1.6.9) so consumers toggle approve mode without hard-coding the system-property string (mirrors LayoutSnapshotAssertions.UPDATE_PROPERTY) - add docs/operations/visual-regression-testing.md (pixel vs semantic, API, approve mode, cross-platform tolerance calibration) - README 'Which API should I use?' gains a pixel-level visual-regression row - add PublicVisualApiDogfoodTest: composes a document and pixel-tests it end-to-end through the public testing.visual surface Full suite 1060/1060 green. --- CHANGELOG.md | 11 ++ README.md | 3 +- docs/operations/visual-regression-testing.md | 139 ++++++++++++++++++ .../testing/visual/PdfVisualRegression.java | 15 +- .../visual/PublicVisualApiDogfoodTest.java | 88 +++++++++++ 5 files changed, 252 insertions(+), 4 deletions(-) create mode 100644 docs/operations/visual-regression-testing.md create mode 100644 src/test/java/com/demcha/testing/visual/PublicVisualApiDogfoodTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 880cf6ad..2c6ee043 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,17 @@ Housekeeping cycle plus the public pixel-level visual-regression API (Track N). consumers can now run the same render-PDF → diff-PNG baseline gate against their own presets and templates instead of copying the harness. Behaviour is unchanged; the PDF→image step is inlined on PDFBox's `PDFRenderer`. +- Exposed `PdfVisualRegression.APPROVE_PROPERTY` (`@since 1.6.9`) — the + `graphcompose.visual.approve` system-property name — so consumers can toggle + baseline-approve mode without hard-coding the string (mirrors + `LayoutSnapshotAssertions.UPDATE_PROPERTY`). + +### Documentation + +- Added [`docs/operations/visual-regression-testing.md`](docs/operations/visual-regression-testing.md): + pixel-vs-semantic guidance, the `PdfVisualRegression` API, approve mode, + baseline layout, and cross-platform tolerance calibration. +- README "Which API should I use?" gains a pixel-level visual-regression row. ## v1.6.8 — 2026-06-01 diff --git a/README.md b/README.md index 04a243bb..85439467 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ Sits between **iText** (low-level page primitives) and **JasperReports** (XML-te - **Server-side PDF generation in Java** — invoices, CVs, reports, proposals, statements, schedules. - **Templated documents from data** — themed presets (`ModernProfessional`, `InvoiceTemplateV2`, …) you parameterise instead of re-styling every time. -- **Regression-tested layouts** — `DocumentSession#layoutSnapshot()` makes layout changes visible in PRs before any byte ships. +- **Regression-tested layouts** — `DocumentSession#layoutSnapshot()` makes layout changes visible in PRs before any byte ships; `PdfVisualRegression` adds a pixel-level gate for font and colour fidelity. - **Streaming PDFs from web backends** — Spring Boot `@RestController` writing straight to the response ([`HttpStreamingExample`](./examples/src/main/java/com/demcha/examples/features/streaming/HttpStreamingExample.java)). - **Higher-level than PDFBox, lighter than JasperReports** — Java DSL describes semantics; no XML templates, no manual coordinates. @@ -90,6 +90,7 @@ GraphCompose uses PDFBox under the hood as the rendering backend — the com | Generate a CV / cover letter from data | Layered templates | `ModernProfessional.create().compose(session, cvDocument)` — see [layered templates](./docs/templates/v2-layered/README.md) | | Add a custom visual primitive | Engine extension | `NodeDefinition` + `PdfFragmentRenderHandler` — see [extension guide](./docs/contributing/extension-guide.md) | | Regression-test generated layouts | Layout snapshots | `DocumentSession#layoutSnapshot()` — quickstart at [Testing your document](./docs/operations/test-your-document.md); full reference at [snapshot testing](./docs/operations/layout-snapshot-testing.md) | +| Pixel-test the rendered PDF (fonts, colours, anti-aliasing) | Visual regression | `PdfVisualRegression.standard()…assertMatchesBaseline(...)` — see [visual regression testing](./docs/operations/visual-regression-testing.md) | | See the live playground / gallery | Next.js showcase site | [Showcase](https://DemchaAV.github.io/GraphCompose/) — source under [`site/`](./site), built with `next build` and deployed via the [Pages workflow](./.github/workflows/deploy-site.yml) | ## Installation diff --git a/docs/operations/visual-regression-testing.md b/docs/operations/visual-regression-testing.md new file mode 100644 index 00000000..980887cd --- /dev/null +++ b/docs/operations/visual-regression-testing.md @@ -0,0 +1,139 @@ +# Visual Regression Testing + +Pixel-level visual regression is the outermost geometry-fidelity layer in GraphCompose. It renders a PDF to one PNG per page and diffs each page against a committed baseline. + +It complements — does not replace — [layout snapshot testing](./layout-snapshot-testing.md): + +1. unit tests validate isolated layout math +2. **layout snapshots** validate the resolved document tree (coordinates, page spans, layer/order) — structural geometry +3. **visual regression** validates the rendered pixels — font shape, colour, anti-aliasing, glyph fallback +4. human inspection of the PDF remains the final eye + +Reach for visual regression when the failure you care about is *pixel-level* rather than *geometry-level*: the layout snapshot still matches but the PDF looks wrong (wrong font, wrong colour, missing glyph, anti-aliasing drift). + +## Pixel vs semantic — which layer? + +| You want to catch… | Use | +|---|---| +| A node moved / page break shifted / sibling order changed | layout snapshot (semantic) | +| The PDF looks identical pixel-for-pixel — fonts, colours, glyphs | **visual regression (pixel)** | +| A specific layout-math rule | a focused unit test | + +The semantic layer is cheap, deterministic, and cross-platform stable. The pixel layer is precise but sensitive to platform font rendering (see [Cross-platform tolerance](#cross-platform-tolerance) below). A flagship template or a preset you publish to others deserves both. + +## Public API + +The harness is `com.demcha.compose.testing.visual.PdfVisualRegression` (`@since 1.6.9`), a sibling to the semantic `com.demcha.compose.testing.layout.*` helpers. It ships in the main artifact, so library consumers use the exact helpers GraphCompose uses in its own tests. + +### Assert a committed baseline + +```java +import com.demcha.compose.GraphCompose; +import com.demcha.compose.document.api.DocumentPageSize; +import com.demcha.compose.document.api.DocumentSession; +import com.demcha.compose.testing.visual.PdfVisualRegression; +import org.junit.jupiter.api.Test; + +class InvoiceVisualParityTest { + + @Test + void invoiceRendersPixelIdentical() throws Exception { + byte[] pdfBytes; + try (DocumentSession document = GraphCompose.document() + .pageSize(DocumentPageSize.A4) + .margin(22, 22, 22, 22) + .create()) { + template.compose(document, spec); + pdfBytes = document.toPdfBytes(); + } + + PdfVisualRegression.standard() + .assertMatchesBaseline("invoice_standard", pdfBytes); + } +} +``` + +`assertMatchesBaseline(name, pdfBytes)` renders every page, compares against `-page-N.png` under the baseline root, and throws `AssertionError` if any page exceeds the configured budget. On failure it writes `-page-N.actual.png` and `-page-N.diff.png` next to the baseline for inspection. + +### Configure the harness + +`PdfVisualRegression` is immutable; every setter returns a copy. + +| Setter | Default | Meaning | +|---|---|---| +| `baselineRoot(Path)` | `src/test/resources/visual-baselines` | where baselines and diff sidecars live | +| `renderScale(float)` | `1.0` | render scale multiplier (`2.0` = retina); must be `> 0` | +| `perPixelTolerance(int)` | `6` | allowed per-channel delta (`0..255`) before a pixel counts as mismatched | +| `mismatchedPixelBudget(long)` | `0` | mismatched pixels tolerated per page before the assertion fails | + +### Diff images directly + +For ad-hoc comparison, render pages and call `ImageDiff` yourself: + +```java +List pages = PdfVisualRegression.standard().renderPages(pdfBytes); +ImageDiff.Result diff = ImageDiff.compare(expectedPng, pages.get(0), 6); +assertThat(diff.withinBudget(0)).isTrue(); +``` + +## Approve mode (blessing baselines) + +There is no baseline the first time. Run with the approve flag to write the current renders as the baseline: + +```bash +./mvnw test -Dtest=InvoiceVisualParityTest -Dgraphcompose.visual.approve=true +``` + +The system-property name is exposed as `PdfVisualRegression.APPROVE_PROPERTY`; the environment variable `GRAPHCOMPOSE_VISUAL_APPROVE=true` works as a fallback. In approve mode the harness writes baselines and skips the diff assertion — so **never enable it in CI verification**, only when you have reviewed the new render and intend to re-bless. + +## Where files live + +- committed baselines: `/-page-N.png` +- mismatch artifacts (normal runs): `/-page-N.actual.png` and `-page-N.diff.png` (mismatched pixels red, matching pixels greyscale) + +Use a flat `name`, or pre-create nested baseline directories — the harness creates the baseline root but not intermediate folders. + +## Cross-platform tolerance + +PDFBox font rasterization drifts slightly across platforms (different system fonts, different rasterizer). A baseline recorded on Windows will not match Linux CI pixel-for-pixel. + +The `standard()` defaults are strict (tolerance `6`, budget `0`) — good for same-platform, deterministic renders. For baselines that must survive a Windows-author → Linux-CI round trip, loosen both. GraphCompose's own CV / cover-letter parity tests calibrate to: + +```java +PdfVisualRegression.standard() + .perPixelTolerance(8) // absorb sub-pixel anti-aliasing drift + .mismatchedPixelBudget(50_000) // ~glyph edges across a full A4 page + .assertMatchesBaseline(slug, pdfBytes); +``` + +Tune these to your fonts and page density: too tight and CI flakes on anti-aliasing noise; too loose and a real regression slips through. Start from the values above and tighten until CI is stable. + +## Using visual regression in downstream projects + +Library consumers use the same published helpers: + +```java +import com.demcha.compose.testing.visual.PdfVisualRegression; + +PdfVisualRegression.standard() + .baselineRoot(Path.of("src", "test", "resources", "pdf-baselines")) + .perPixelTolerance(8) + .mismatchedPixelBudget(50_000) + .assertMatchesBaseline("reports/monthly_invoice", pdfBytes); +``` + +`PublicVisualApiDogfoodTest` in this repository drives exactly this consumer workflow end-to-end and proves the published surface is sufficient without any package-private access. + +## When not to use pixel regression + +- when structural geometry is what you care about → use a [layout snapshot](./layout-snapshot-testing.md) (cheaper, cross-platform stable) +- when a small unit test proves the same rule more directly +- as the *only* gate on a CI that runs on a different OS than where baselines were recorded — pair it with semantic snapshots and a sensible tolerance budget + +## Examples in this repository + +- `CvV2VisualParityTest`, `CoverLetterV2VisualParityTest` — preset parity with Windows-baseline / Linux-CI calibration +- `ShapeContainerVisualRegressionTest` — engine primitive fidelity +- `TableRowSpanDemoTest` — table rendering +- `PdfVisualRegressionTest` — the harness's own unit tests +- `PublicVisualApiDogfoodTest` — consumer-surface dogfood diff --git a/src/main/java/com/demcha/compose/testing/visual/PdfVisualRegression.java b/src/main/java/com/demcha/compose/testing/visual/PdfVisualRegression.java index 717da220..e3e2aa09 100644 --- a/src/main/java/com/demcha/compose/testing/visual/PdfVisualRegression.java +++ b/src/main/java/com/demcha/compose/testing/visual/PdfVisualRegression.java @@ -40,7 +40,16 @@ */ public final class PdfVisualRegression { - private static final String APPROVE_SYS_PROP = "graphcompose.visual.approve"; + /** + * System property that switches the harness into approve mode + * ({@code -Dgraphcompose.visual.approve=true}): instead of asserting, it + * (re)writes the current renders to the baseline location and skips the + * diff. The environment variable {@code GRAPHCOMPOSE_VISUAL_APPROVE=true} + * works as a fallback. + * + * @since 1.6.9 + */ + public static final String APPROVE_PROPERTY = "graphcompose.visual.approve"; private static final String APPROVE_ENV_VAR = "GRAPHCOMPOSE_VISUAL_APPROVE"; private final Path baselineRoot; @@ -155,7 +164,7 @@ public void assertMatchesBaseline(String baselineName, byte[] pdfBytes) throws I Path actualOut = sidecarPath(baselineName, page, "actual"); ImageIO.write(rendered.get(page), "png", actualOut.toFile()); failures.add("Missing baseline " + baseline + " — wrote rendered output to " + actualOut - + ". Re-run with -D" + APPROVE_SYS_PROP + "=true to approve."); + + ". Re-run with -D" + APPROVE_PROPERTY + "=true to approve."); continue; } BufferedImage expected = ImageIO.read(baseline.toFile()); @@ -206,7 +215,7 @@ private Path sidecarPath(String baselineName, int pageIndex, String suffix) { } private static boolean approveMode() { - String prop = System.getProperty(APPROVE_SYS_PROP); + String prop = System.getProperty(APPROVE_PROPERTY); if (prop != null) { return Boolean.parseBoolean(prop); } diff --git a/src/test/java/com/demcha/testing/visual/PublicVisualApiDogfoodTest.java b/src/test/java/com/demcha/testing/visual/PublicVisualApiDogfoodTest.java new file mode 100644 index 00000000..97f617fb --- /dev/null +++ b/src/test/java/com/demcha/testing/visual/PublicVisualApiDogfoodTest.java @@ -0,0 +1,88 @@ +package com.demcha.testing.visual; + +import com.demcha.compose.GraphCompose; +import com.demcha.compose.document.api.DocumentPageSize; +import com.demcha.compose.document.api.DocumentSession; +import com.demcha.compose.document.style.DocumentTextStyle; +import com.demcha.compose.testing.visual.PdfVisualRegression; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.nio.file.Path; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Drives a consumer-style "compose a document, then pixel-test the rendered + * PDF" workflow entirely through the public {@code + * com.demcha.compose.testing.visual} surface — no package-private access. + * + *

Sibling to {@code LayoutSnapshotPublicApiDogfoodTest} on the semantic + * layer; it proves the published harness is sufficient for downstream adoption + * and guards against accidental private-to-public regressions in CI.

+ */ +class PublicVisualApiDogfoodTest { + + @TempDir + Path tempDir; + + @Test + void shouldSupportConsumerPixelRegressionThroughPublicApi() throws Exception { + Path baselineRoot = tempDir.resolve("pdf-baselines"); + String baselineName = "consumer_simple_document"; + byte[] pdfBytes = renderConsumerDocument(); + + PdfVisualRegression harness = PdfVisualRegression.standard().baselineRoot(baselineRoot); + + // 1. Consumer blesses a fresh baseline via the public approve flag. + String previous = System.getProperty(PdfVisualRegression.APPROVE_PROPERTY); + try { + System.setProperty(PdfVisualRegression.APPROVE_PROPERTY, "true"); + harness.assertMatchesBaseline(baselineName, pdfBytes); + } finally { + restoreProperty(PdfVisualRegression.APPROVE_PROPERTY, previous); + } + + assertThat(baselineRoot.resolve(baselineName + "-page-0.png")) + .as("approve mode should write a page-0 baseline") + .exists() + .isRegularFile(); + + // 2. A second render of the same document matches the just-written baseline. + harness.assertMatchesBaseline(baselineName, pdfBytes); + + // 3. A matching run must not leave a mismatch sidecar. + assertThat(baselineRoot.resolve(baselineName + "-page-0.actual.png")) + .as("a matching run must not leave a mismatch sidecar") + .doesNotExist(); + } + + private byte[] renderConsumerDocument() throws Exception { + try (DocumentSession document = GraphCompose.document() + .pageSize(DocumentPageSize.A4) + .margin(18, 18, 18, 18) + .create()) { + document.dsl() + .pageFlow() + .name("ConsumerRoot") + .spacing(8) + .addParagraph(paragraph -> paragraph + .name("Title") + .text("Consumer pixel baseline") + .textStyle(DocumentTextStyle.DEFAULT)) + .addShape(shape -> shape + .name("AccentBox") + .size(120, 32)) + .build(); + return document.toPdfBytes(); + } + } + + private void restoreProperty(String key, String previous) { + if (previous == null) { + System.clearProperty(key); + } else { + System.setProperty(key, previous); + } + } +}