Skip to content

perf(tests): the sample projects publish once per family, not once per sample - #6722

Merged
delchev merged 1 commit into
masterfrom
perf/it-consolidate-samples
Aug 13, 2026
Merged

perf(tests): the sample projects publish once per family, not once per sample#6722
delchev merged 1 commit into
masterfrom
perf/it-consolidate-samples

Conversation

@delchev

@delchev delchev commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Item 1 of #6710: the sample-project ITs were the same journey 14 times - clone a dirigiblelabs/* repo through the Git perspective, publishAll, verify - each paying its own Dirigible boot, Chrome session and publish cycle (1088s on the h2 leg of run 31679871168, the biggest remaining block in the samples shard).

They now clone a family of repos into one workspace, publish once, and verify one sample per @Test method: 13 boots become 2.

  • SampleProjectsIT (replaces SampleProjectRepositoryIT) clones a list of repos, publishes once, force-syncs and waits for a stable synchronization - all in one @BeforeAll under @TestInstance(PER_CLASS) (so the hook can use the injected IDE) and @DirtiesContext(AFTER_CLASS) (so the family shares the boot).
  • TypeScriptSampleProjectsIT - the 8 TS/JS decorator samples, 8 tests.
  • JavaSampleProjectsIT - the 5 client-Java samples, 5 tests.
  • SampleLibraryLocalNativeAppIT moves onto the new base but keeps its own instance: it spawns a real OS process (npm install on first run) and grants a role on the fly.

Every assertion is carried over unchanged.

Why two families and not one class

The preflight #6710 asked for (no duplicate FQNs across the sample-java-* repos, since the family shares one javac batch and one ClientClassLoader) came back clean - all 17 client classes sit in distinct demo.* packages with distinct simple names, so no bean-name collision either.

It surfaced a different collision the issue had not anticipated: sample-entity-decorators and sample-java-entity-decorators both own the SAMPLE_COUNTRY table (identical .table, UNIQUE on three columns) and both CSVIM-seed it from their own countries.csv. Publishing them side by side means two .table artefacts claiming one name and two importers seeding it. Splitting along the TS/Java line - the alternative the issue offers - keeps them in separate instances and needs no change to any sample repo.

Checked for the rest of the collision surface too: within each family the table names, listener destinations (sample-listener-decorator/OrderListener vs java-invoice-queue), websocket endpoints (order-ws vs java-chat/java-ticker) and job crons are all distinct, and CountryController.ts is the only @Controller in the TS family - which is why the exact-match OpenAPI assertion survives seven more samples sharing the instance.

Verification

Both consolidated classes green locally, headless:

TypeScriptSampleProjectsIT   Tests run: 8, Failures: 0, Errors: 0   109.3 s
JavaSampleProjectsIT         Tests run: 5, Failures: 0, Errors: 0    91.0 s

That is 13 sample verifications in ~200s of test time on 2 boots, against 1088s for the 14 per-sample classes on the h2 leg of run 31679871168. Two failures on the way here, both fixed and both worth knowing about:

  1. @TestInstance(PER_CLASS) is incompatible with this test framework. The obvious shape for "clone and publish once" is a non-static @BeforeAll, which needs PER_CLASS to reach the injected IDE. But PER_CLASS creates and autowires the test instance BEFORE @BeforeAll runs, so the Spring context - the whole platform - starts before IntegrationTest.cleanBeforeTestClassExecution() deletes the Dirigible folder. The next synchronization pass then reaps the platform's own registry:
    SynchronizationProcessor - Definition deleted: /shell-ide/extensions/shell.extension
    
    and the IDE renders with no perspectives at all - the failure surfaces as perspective-git not found against a blank page, nowhere near its cause. The publish is now a @BeforeEach guarded by a static, which keeps the cleaner ahead of the boot.
  2. The one browser-driven verification needed its own session. The base closes the driver after each test, so websocketDecorator gets a fresh Chrome whenever it is not the first method to run, and the sample's page served the login form instead. It now opens the home page first. Order-independence is a requirement of this design, not a detail - the base javadoc says so.

Notes for the reader

  • The verifications must stay order-independent; they run against one live instance. The LogsAsserters are created inside each test (an asserter only sees messages logged after it attaches) and every awaited message comes from a recurring cron or from a trigger the test itself fires.
  • SampleLibraryLocalNativeAppIT's @BeforeAll that raises the native-app ready timeout now runs after the base's clone-and-publish. That is soon enough: the artefact declares "mode": "lazy" and only a StartMode.ALWAYS app is spawned by the bootstrap, the monitor job or the synchronizer - this one waits for the first proxied request, which the test method makes.
  • @Tag("sample") stays on the base, so the samples CI shard selector (sample | camel) is unchanged.
  • Docs updated to match: the "Sample-project tests" bullet in CLAUDE.md now documents the family model, the collision rule and the order-independence requirement; stale class names swept from CLAUDE_FEATURES.md, engine-intent/engine-java/engine-native-apps CLAUDE.md and two javadocs.

Remaining #6710 items (camel ITs to HTTP-level, shard rebalance, PG-leg scope) are untouched.

…r sample

The 14 sample-project ITs were the same journey 14 times - clone a
dirigiblelabs/* repo through the Git perspective, publishAll, verify - each
paying its own Dirigible boot, Chrome session and publish cycle (1088s on the
h2 leg, the biggest remaining block in the samples shard).

SampleProjectsIT now clones a FAMILY of repos into one workspace, publishes
once, and lets each subclass verify one sample per @test under AFTER_CLASS:
TypeScriptSampleProjectsIT (8 samples) and JavaSampleProjectsIT (5 samples)
replace 13 classes, so 13 boots become 2. SampleLibraryLocalNativeAppIT moves
onto the new base but keeps its own instance - it spawns a real OS process.
Every assertion is carried over unchanged.

Two families rather than one class because sample-entity-decorators and
sample-java-entity-decorators both own the SAMPLE_COUNTRY table and both
CSVIM-seed it; the split keeps them in separate instances and needs no change
to any sample repo. The preflight for the Java family came back clean - all 17
client classes sit in distinct demo.* packages with distinct simple names, so
the shared javac batch and bean container have nothing to collide on.

The one-time publish is a guarded @beforeeach, not a @BeforeAll on a
@testinstance(PER_CLASS) class: PER_CLASS autowires the instance - booting the
platform - before @BeforeAll runs, so IntegrationTest's cleaner then deleted
the Dirigible folder under the running instance and the next sync pass reaped
the platform's own registry (Definition deleted:
/shell-ide/extensions/shell.extension), leaving the IDE with no perspectives.
The guard is keyed by class, not a boolean: the field is shared by every
subclass, and a flag would let the second family skip its own publish.

Part of #6710.
@delchev
delchev merged commit 1f69fef into master Aug 13, 2026
8 of 9 checks passed
@delchev
delchev deleted the perf/it-consolidate-samples branch August 13, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant