Skip to content

ci(tests): shard the integration-test jobs into api / ui / samples slices - #6702

Merged
delchev merged 1 commit into
masterfrom
ci/shard-integration-tests
Aug 13, 2026
Merged

ci(tests): shard the integration-test jobs into api / ui / samples slices#6702
delchev merged 1 commit into
masterfrom
ci/shard-integration-tests

Conversation

@delchev

@delchev delchev commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Problem

The full integration-test suite (116 IT classes, 275 test methods) runs strictly sequentially, so each DB leg of build.yml / nightly.yml takes 2h+ wall clock (e.g. run 31679871168: h2 leg 2h06m, postgresql leg 2h30m). The Maven build phase is only ~3 min — everything else is failsafe.

Change

Split every DB leg into three parallel matrix shards selected by JUnit 5 tag expressions (the same mechanism the PR smoke job already uses):

shard selector classes measured time
api -Dit.groups='!ui' 57 ~48 min
ui -Dit.groups='ui & !sample & !camel' 34 ~48 min
samples -Dit.groups='sample | camel' 25 ~27 min

The run's wall clock becomes the slowest shard (~50 min, meeting the ≤1h goal) instead of the whole suite, and a red shard immediately names the failing family. Total compute stays the same (+~3 min build phase per extra shard).

New tags:

  • @Tag("sample") on the SampleProjectRepositoryIT base — inherited by all 14 sample-project ITs.
  • @Tag("camel") on each of the 11 ITs in ui/tests/camel — their PredefinedProjectIT base is shared with non-camel tests, so the base cannot carry the tag.

Also: per-shard timeout-minutes: 90 (was 150), per-shard screenshot artifact names, fail-fast: false so one red shard doesn't cancel the others, and doc updates (CLAUDE.md CI reference, tests-integrations pom comment).

Verification

  • Shard partition proven via JUnit Platform discovery over the compiled test classes: 57 + 34 + 25 = 116 classes — exactly the full-suite discovery count, with zero classes selected by more than one shard.
  • mvn compile -pl tests/tests-integrations and mvn formatter:validate -pl tests/tests-integrations green; both workflow files YAML-validated.
  • The timing figures come from the per-class failsafe timings of the linked run's h2 leg.

Follow-ups (separate PRs, discussed in the analysis)

  1. Relax @DirtiesContext(AFTER_EACH_TEST_METHOD)AFTER_CLASS class-by-class (275 → ~116 Spring boots, ~25–35 min/leg of compute; DatabaseFacadeIT already proves the pattern: 18 tests in 0.9 s).
  2. Consolidate duplicate journeys (the 14 sample-project ITs share one clone→publish→verify flow; the BPMN editor trio; CreateNewFileIT spends 431 s looping every file type through the UI).
  3. Consider running the PG leg with the DB-sensitive shards only.

Note: the linked run's PG leg also failed functionally (CsvProcessorIT, CsvimReimportITTable metadata was not found ... in schema [null], PostgreSQL-specific) — that is independent of this change and deserves its own issue.

🤖 Generated with Claude Code

…ices

The full IT suite runs its 116 classes strictly sequentially, so each DB
leg of build.yml/nightly.yml took 2h+ wall clock. Split every leg into
three parallel matrix shards selected by JUnit 5 tag expressions:

  api     -Dit.groups='!ui'                  (57 classes, ~48 min)
  ui      -Dit.groups='ui & !sample & !camel' (34 classes, ~48 min)
  samples -Dit.groups='sample | camel'        (25 classes, ~27 min)

The run's wall clock becomes the slowest shard (~50 min) instead of the
whole suite. The shards partition the suite - verified via JUnit
Platform discovery: 57+34+25 = 116 classes, zero overlaps.

New tags: @tag("sample") on the SampleProjectRepositoryIT base
(inherited by all sample-project ITs) and @tag("camel") on each IT in
ui/tests/camel (their PredefinedProjectIT base is shared with non-camel
tests, so it cannot carry the tag). Shard timeouts drop from 150 to 90
minutes; failure screenshots are uploaded per shard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant