Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,17 @@ Housekeeping cycle plus the public pixel-level visual-regression API (Track N).
consumers can now run the same render-PDF → diff-PNG baseline gate against
their own presets and templates instead of copying the harness. Behaviour is
unchanged; the PDF→image step is inlined on PDFBox's `PDFRenderer`.
- Exposed `PdfVisualRegression.APPROVE_PROPERTY` (`@since 1.6.9`) — the
`graphcompose.visual.approve` system-property name — so consumers can toggle
baseline-approve mode without hard-coding the string (mirrors
`LayoutSnapshotAssertions.UPDATE_PROPERTY`).

### Documentation

- Added [`docs/operations/visual-regression-testing.md`](docs/operations/visual-regression-testing.md):
pixel-vs-semantic guidance, the `PdfVisualRegression` API, approve mode,
baseline layout, and cross-platform tolerance calibration.
- README "Which API should I use?" gains a pixel-level visual-regression row.

## v1.6.8 — 2026-06-01

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Sits between **iText** (low-level page primitives) and **JasperReports** (XML-te

- **Server-side PDF generation in Java** — invoices, CVs, reports, proposals, statements, schedules.
- **Templated documents from data** — themed presets (`ModernProfessional`, `InvoiceTemplateV2`, …) you parameterise instead of re-styling every time.
- **Regression-tested layouts** — `DocumentSession#layoutSnapshot()` makes layout changes visible in PRs before any byte ships.
- **Regression-tested layouts** — `DocumentSession#layoutSnapshot()` makes layout changes visible in PRs before any byte ships; `PdfVisualRegression` adds a pixel-level gate for font and colour fidelity.
- **Streaming PDFs from web backends** — Spring Boot `@RestController` writing straight to the response ([`HttpStreamingExample`](./examples/src/main/java/com/demcha/examples/features/streaming/HttpStreamingExample.java)).
- **Higher-level than PDFBox, lighter than JasperReports** — Java DSL describes semantics; no XML templates, no manual coordinates.

Expand Down Expand Up @@ -90,6 +90,7 @@ GraphCompose uses PDFBox under the hood as the rendering backend — the com
| Generate a CV / cover letter from data | Layered templates | `ModernProfessional.create().compose(session, cvDocument)` — see [layered templates](./docs/templates/v2-layered/README.md) |
| Add a custom visual primitive | Engine extension | `NodeDefinition` + `PdfFragmentRenderHandler` — see [extension guide](./docs/contributing/extension-guide.md) |
| Regression-test generated layouts | Layout snapshots | `DocumentSession#layoutSnapshot()` — quickstart at [Testing your document](./docs/operations/test-your-document.md); full reference at [snapshot testing](./docs/operations/layout-snapshot-testing.md) |
| Pixel-test the rendered PDF (fonts, colours, anti-aliasing) | Visual regression | `PdfVisualRegression.standard()…assertMatchesBaseline(...)` — see [visual regression testing](./docs/operations/visual-regression-testing.md) |
| See the live playground / gallery | Next.js showcase site | [Showcase](https://DemchaAV.github.io/GraphCompose/) — source under [`site/`](./site), built with `next build` and deployed via the [Pages workflow](./.github/workflows/deploy-site.yml) |

## Installation
Expand Down
139 changes: 139 additions & 0 deletions docs/operations/visual-regression-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# Visual Regression Testing

Pixel-level visual regression is the outermost geometry-fidelity layer in GraphCompose. It renders a PDF to one PNG per page and diffs each page against a committed baseline.

It complements — does not replace — [layout snapshot testing](./layout-snapshot-testing.md):

1. unit tests validate isolated layout math
2. **layout snapshots** validate the resolved document tree (coordinates, page spans, layer/order) — structural geometry
3. **visual regression** validates the rendered pixels — font shape, colour, anti-aliasing, glyph fallback
4. human inspection of the PDF remains the final eye

Reach for visual regression when the failure you care about is *pixel-level* rather than *geometry-level*: the layout snapshot still matches but the PDF looks wrong (wrong font, wrong colour, missing glyph, anti-aliasing drift).

## Pixel vs semantic — which layer?

| You want to catch… | Use |
|---|---|
| A node moved / page break shifted / sibling order changed | layout snapshot (semantic) |
| The PDF looks identical pixel-for-pixel — fonts, colours, glyphs | **visual regression (pixel)** |
| A specific layout-math rule | a focused unit test |

The semantic layer is cheap, deterministic, and cross-platform stable. The pixel layer is precise but sensitive to platform font rendering (see [Cross-platform tolerance](#cross-platform-tolerance) below). A flagship template or a preset you publish to others deserves both.

## Public API

The harness is `com.demcha.compose.testing.visual.PdfVisualRegression` (`@since 1.6.9`), a sibling to the semantic `com.demcha.compose.testing.layout.*` helpers. It ships in the main artifact, so library consumers use the exact helpers GraphCompose uses in its own tests.

### Assert a committed baseline

```java
import com.demcha.compose.GraphCompose;
import com.demcha.compose.document.api.DocumentPageSize;
import com.demcha.compose.document.api.DocumentSession;
import com.demcha.compose.testing.visual.PdfVisualRegression;
import org.junit.jupiter.api.Test;

class InvoiceVisualParityTest {

@Test
void invoiceRendersPixelIdentical() throws Exception {
byte[] pdfBytes;
try (DocumentSession document = GraphCompose.document()
.pageSize(DocumentPageSize.A4)
.margin(22, 22, 22, 22)
.create()) {
template.compose(document, spec);
pdfBytes = document.toPdfBytes();
}

PdfVisualRegression.standard()
.assertMatchesBaseline("invoice_standard", pdfBytes);
}
}
```

`assertMatchesBaseline(name, pdfBytes)` renders every page, compares against `<name>-page-N.png` under the baseline root, and throws `AssertionError` if any page exceeds the configured budget. On failure it writes `<name>-page-N.actual.png` and `<name>-page-N.diff.png` next to the baseline for inspection.

### Configure the harness

`PdfVisualRegression` is immutable; every setter returns a copy.

| Setter | Default | Meaning |
|---|---|---|
| `baselineRoot(Path)` | `src/test/resources/visual-baselines` | where baselines and diff sidecars live |
| `renderScale(float)` | `1.0` | render scale multiplier (`2.0` = retina); must be `> 0` |
| `perPixelTolerance(int)` | `6` | allowed per-channel delta (`0..255`) before a pixel counts as mismatched |
| `mismatchedPixelBudget(long)` | `0` | mismatched pixels tolerated per page before the assertion fails |

### Diff images directly

For ad-hoc comparison, render pages and call `ImageDiff` yourself:

```java
List<BufferedImage> pages = PdfVisualRegression.standard().renderPages(pdfBytes);
ImageDiff.Result diff = ImageDiff.compare(expectedPng, pages.get(0), 6);
assertThat(diff.withinBudget(0)).isTrue();
```

## Approve mode (blessing baselines)

There is no baseline the first time. Run with the approve flag to write the current renders as the baseline:

```bash
./mvnw test -Dtest=InvoiceVisualParityTest -Dgraphcompose.visual.approve=true
```

The system-property name is exposed as `PdfVisualRegression.APPROVE_PROPERTY`; the environment variable `GRAPHCOMPOSE_VISUAL_APPROVE=true` works as a fallback. In approve mode the harness writes baselines and skips the diff assertion — so **never enable it in CI verification**, only when you have reviewed the new render and intend to re-bless.

## Where files live

- committed baselines: `<baselineRoot>/<name>-page-N.png`
- mismatch artifacts (normal runs): `<baselineRoot>/<name>-page-N.actual.png` and `<name>-page-N.diff.png` (mismatched pixels red, matching pixels greyscale)

Use a flat `name`, or pre-create nested baseline directories — the harness creates the baseline root but not intermediate folders.

## Cross-platform tolerance

PDFBox font rasterization drifts slightly across platforms (different system fonts, different rasterizer). A baseline recorded on Windows will not match Linux CI pixel-for-pixel.

The `standard()` defaults are strict (tolerance `6`, budget `0`) — good for same-platform, deterministic renders. For baselines that must survive a Windows-author → Linux-CI round trip, loosen both. GraphCompose's own CV / cover-letter parity tests calibrate to:

```java
PdfVisualRegression.standard()
.perPixelTolerance(8) // absorb sub-pixel anti-aliasing drift
.mismatchedPixelBudget(50_000) // ~glyph edges across a full A4 page
.assertMatchesBaseline(slug, pdfBytes);
```

Tune these to your fonts and page density: too tight and CI flakes on anti-aliasing noise; too loose and a real regression slips through. Start from the values above and tighten until CI is stable.

## Using visual regression in downstream projects

Library consumers use the same published helpers:

```java
import com.demcha.compose.testing.visual.PdfVisualRegression;

PdfVisualRegression.standard()
.baselineRoot(Path.of("src", "test", "resources", "pdf-baselines"))
.perPixelTolerance(8)
.mismatchedPixelBudget(50_000)
.assertMatchesBaseline("reports/monthly_invoice", pdfBytes);
```

`PublicVisualApiDogfoodTest` in this repository drives exactly this consumer workflow end-to-end and proves the published surface is sufficient without any package-private access.

## When not to use pixel regression

- when structural geometry is what you care about → use a [layout snapshot](./layout-snapshot-testing.md) (cheaper, cross-platform stable)
- when a small unit test proves the same rule more directly
- as the *only* gate on a CI that runs on a different OS than where baselines were recorded — pair it with semantic snapshots and a sensible tolerance budget

## Examples in this repository

- `CvV2VisualParityTest`, `CoverLetterV2VisualParityTest` — preset parity with Windows-baseline / Linux-CI calibration
- `ShapeContainerVisualRegressionTest` — engine primitive fidelity
- `TableRowSpanDemoTest` — table rendering
- `PdfVisualRegressionTest` — the harness's own unit tests
- `PublicVisualApiDogfoodTest` — consumer-surface dogfood
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,16 @@
*/
public final class PdfVisualRegression {

private static final String APPROVE_SYS_PROP = "graphcompose.visual.approve";
/**
* System property that switches the harness into approve mode
* ({@code -Dgraphcompose.visual.approve=true}): instead of asserting, it
* (re)writes the current renders to the baseline location and skips the
* diff. The environment variable {@code GRAPHCOMPOSE_VISUAL_APPROVE=true}
* works as a fallback.
*
* @since 1.6.9
*/
public static final String APPROVE_PROPERTY = "graphcompose.visual.approve";
private static final String APPROVE_ENV_VAR = "GRAPHCOMPOSE_VISUAL_APPROVE";

private final Path baselineRoot;
Expand Down Expand Up @@ -155,7 +164,7 @@ public void assertMatchesBaseline(String baselineName, byte[] pdfBytes) throws I
Path actualOut = sidecarPath(baselineName, page, "actual");
ImageIO.write(rendered.get(page), "png", actualOut.toFile());
failures.add("Missing baseline " + baseline + " — wrote rendered output to " + actualOut
+ ". Re-run with -D" + APPROVE_SYS_PROP + "=true to approve.");
+ ". Re-run with -D" + APPROVE_PROPERTY + "=true to approve.");
continue;
}
BufferedImage expected = ImageIO.read(baseline.toFile());
Expand Down Expand Up @@ -206,7 +215,7 @@ private Path sidecarPath(String baselineName, int pageIndex, String suffix) {
}

private static boolean approveMode() {
String prop = System.getProperty(APPROVE_SYS_PROP);
String prop = System.getProperty(APPROVE_PROPERTY);
if (prop != null) {
return Boolean.parseBoolean(prop);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
package com.demcha.testing.visual;

import com.demcha.compose.GraphCompose;
import com.demcha.compose.document.api.DocumentPageSize;
import com.demcha.compose.document.api.DocumentSession;
import com.demcha.compose.document.style.DocumentTextStyle;
import com.demcha.compose.testing.visual.PdfVisualRegression;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;

import java.nio.file.Path;

import static org.assertj.core.api.Assertions.assertThat;

/**
* Drives a consumer-style "compose a document, then pixel-test the rendered
* PDF" workflow entirely through the public {@code
* com.demcha.compose.testing.visual} surface — no package-private access.
*
* <p>Sibling to {@code LayoutSnapshotPublicApiDogfoodTest} on the semantic
* layer; it proves the published harness is sufficient for downstream adoption
* and guards against accidental private-to-public regressions in CI.</p>
*/
class PublicVisualApiDogfoodTest {

@TempDir
Path tempDir;

@Test
void shouldSupportConsumerPixelRegressionThroughPublicApi() throws Exception {
Path baselineRoot = tempDir.resolve("pdf-baselines");
String baselineName = "consumer_simple_document";
byte[] pdfBytes = renderConsumerDocument();

PdfVisualRegression harness = PdfVisualRegression.standard().baselineRoot(baselineRoot);

// 1. Consumer blesses a fresh baseline via the public approve flag.
String previous = System.getProperty(PdfVisualRegression.APPROVE_PROPERTY);
try {
System.setProperty(PdfVisualRegression.APPROVE_PROPERTY, "true");
harness.assertMatchesBaseline(baselineName, pdfBytes);
} finally {
restoreProperty(PdfVisualRegression.APPROVE_PROPERTY, previous);
}

assertThat(baselineRoot.resolve(baselineName + "-page-0.png"))
.as("approve mode should write a page-0 baseline")
.exists()
.isRegularFile();

// 2. A second render of the same document matches the just-written baseline.
harness.assertMatchesBaseline(baselineName, pdfBytes);

// 3. A matching run must not leave a mismatch sidecar.
assertThat(baselineRoot.resolve(baselineName + "-page-0.actual.png"))
.as("a matching run must not leave a mismatch sidecar")
.doesNotExist();
}

private byte[] renderConsumerDocument() throws Exception {
try (DocumentSession document = GraphCompose.document()
.pageSize(DocumentPageSize.A4)
.margin(18, 18, 18, 18)
.create()) {
document.dsl()
.pageFlow()
.name("ConsumerRoot")
.spacing(8)
.addParagraph(paragraph -> paragraph
.name("Title")
.text("Consumer pixel baseline")
.textStyle(DocumentTextStyle.DEFAULT))
.addShape(shape -> shape
.name("AccentBox")
.size(120, 32))
.build();
return document.toPdfBytes();
}
}

private void restoreProperty(String key, String previous) {
if (previous == null) {
System.clearProperty(key);
} else {
System.setProperty(key, previous);
}
}
}