Skip to content

fix(ci): drop dead build artifact, guard an NPE in a catch block, and make integration tests report their failures - #36943

Open
wezell wants to merge 14 commits into
mainfrom
issue-36940-ci-speedup
Open

fix(ci): drop dead build artifact, guard an NPE in a catch block, and make integration tests report their failures#36943
wezell wants to merge 14 commits into
mainfrom
issue-36940-ci-speedup

Conversation

@wezell

@wezell wezell commented Aug 7, 2026

Copy link
Copy Markdown
Member

Fixes #36942 (partially — the shard rebalance is deferred, see below).

A set of independent CI and test fixes, none of which depend on each other.

Build

Dead build-classes artifact removed. It was uploaded on every build and downloaded by nobody — the only restore-classes: true caller is deploy-javadoc, which no workflow invokes. Costs ~0.5m on the serial critical path that gates all ~25 test jobs.

A production bug

AWSS3PublishingEndPoint.validatePublishingEndPoint() catches S3 connection failures to report them to the user, then builds that notification with PortalUtil.getUser().getUserId(). But:

final HttpServletRequest req = HttpServletRequestThreadLocal.INSTANCE.getRequest();
return req == null ? null : getUser(req);

Null whenever no request is bound to the thread — scheduled jobs, background tasks, tests. So the error handler NPE'd, replacing the S3 error it was trying to report and defeating the catch entirely. Now null-guarded.

Tests that could not report their own failures

A scan of all 845 integration test files found 18 catch blocks that fail a test while discarding the exception:

catch (Exception e) { Assert.fail("Should work"); }

Surefire retries then produce N identical, contentless messages. This cost a full CI run to diagnose on PublishingEndPointTest — four retries, four copies of "No Exception should be thrown", no way to tell what threw. Two were worse: a bare Assert.fail() with no message at all.

All 18 now throw new AssertionError(msg, e) — same failure, cause chained. The scanner is included as .github/scripts/test-balance/find_swallowed.py; a re-scan reports zero remaining. For balance, 90 catch blocks already did this correctly — these were the outliers.

Test performance

before change
Task240306...Test 7.26m teardown batched: find+destroy per inode over ~150 Language Variables × 3 tests, each its own transaction → one query, one transaction
ImportUtilTest 34s of Thread.sleep → awaitility; the 30s one slept before its first check, so it always paid full cost
BrowserAPITest 4.06m 145 file assets → 36; the assertions depend on page-size relationships, not absolute counts

FolderResourceTest got the same treatment, but only where safe — three of its five large loops are load-bearing (they pin the former 20-item cap, the @DefaultValue("40") limit, and that limit=-1 bypasses it). Shrinking those would make them pass trivially.

Also removed Task240306's deliberate delete of the Language Variable content type — see #36958, which tracks marking it system so that delete becomes impossible.

Why the shard rebalance is not here

It works. Measured on run 31203516115 with fail-fast disabled so every shard reported:

IT shards    22.2 - 26.2m   (was 38.9m)
Postman      18.6 - 28.8m   (was 37.8m)

But 3 of 7 IT shards and 5 of 9 Postman shards failed — all tests that only passed because of which suite-mates ran before them.

The Postman group names turned out not to be labels but dependency clusters: category-content grouped Category + ContentResourceV1 + Content_Resource because those collections share data. Rebalancing on time alone shattered them.

Preserved on issue-36942-shard-rebalance-wip (f612b27) so it can be resumed rather than rebuilt. It is worth resuming — it is an effective detector of tests that depend on their neighbours, and every failure it surfaced was a real defect, including the production NPE above and the deletable content type in #36958. But that is a test-independence programme, and it should not gate these fixes.

Verification

Both :dotcms-core and :dotcms-integration compile. Not run locally — needs the full IT stack.

wezell and others added 6 commits August 5, 2026 21:35
…36914)

Category Content ran ~40m with ContentTypeResourceTests alone taking 14m37s
of it, while the Template job finished in ~10m (mostly boot overhead).
Rebalancing the collection across existing groups cuts the Postman critical
path ~15m without adding a job or paying another ~12m dotCMS boot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ained (#36914)

The 'ensure' tests assert a Video content type exists, but nothing creates
it eagerly at startup — it only existed because collections that previously
ran before this one in the category-content group created it indirectly.
Moving the collection to the template group exposed this (Video missing,
dotAsset present). Add a create-if-missing setup request so the collection
passes regardless of group placement or ordering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… artifact

The PR pipeline's wall clock is bounded by its slowest test job. Two jobs were
tied at the top -- Integration MainSuite 1a (38.9m) and Postman GraphQL (37.8m)
-- so rebalancing either one alone moved nothing. This rebalances both, plus
removes an artifact that was uploaded on every build and consumed by nothing.

Measured from GitHub Actions job timings and failsafe/newman reports:

  Integration  7 shards, 130.8m of test time, max 38.9m, ideal 26.5m
  Postman     11 shards, 147.8m of test time, max 37.8m, ideal 22.8m
  fixed cost per shard: ~7.6m (IT), ~9.4m (Postman)

Changes:

* Integration: repack all 557 classes across 7 MainSuite shards by measured
  per-class time (LPT bin-packing) instead of by accretion. Every shard is now
  18.4m of test time (spread 0.0m), vs 14.2m-31.1m before. Adds MainSuite3b and
  MainSuite4a. Class count is a poor proxy for time and was the reason the old
  split drifted -- one shard held 298 classes/31m, another 52 classes/14m.
  Suites now use fully-qualified class names so rebalancing does not churn
  imports. The "run FIRST on purpose" full-scan block from #36911 is preserved
  at the head of each shard.

* Postman: regroup on measured newman time, 11 shards -> 9, all at 16.4m.
  GraphQLTests was a single 25.6m collection and the binding constraint on the
  whole Postman tail -- no regrouping could get below 25.6m + 9.4m overhead.
  It is now folder-sharded via a new `folders` key in config.json rather than
  by splitting the 518KB collection file. Four micro-groups (ai, pp, container,
  template) that were ~85% fixed overhead are merged away.

* index.js: support the `folders` key, and hard-fail on a folder name that does
  not exist. Newman runs zero requests for an unknown folder and still exits
  green, so a typo would silently delete coverage. `errors` never affected the
  exit code -- only `failures` did -- so this validates up front and exits 1,
  consistent with how an unknown groupname is already handled.

* verify-config.js: checks config.json against the collections on disk --
  missing collections, unknown folders, double-claimed collections, and folders
  covered by no shard. It caught a real bug while writing this: a folder named
  "Related content with condition / query" contains a slash, and deriving names
  by splitting report labels on " / " silently truncated it.

* maven-job: drop the `build-classes` artifact. It was uploaded on every build
  (~0.5m on the serial critical path, since the build gates every test job) and
  downloaded by nobody -- the only `restore-classes: true` caller is
  deploy-javadoc, which no workflow invokes.

Net job count is unchanged: integration 7 -> 9 shards, postman 11 -> 9.

Expected tail: 38.9m -> ~26m. Wall clock above that floor is queue time, which
this does not address.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH4Pes5S9d1AQorJBNeFJU
@claude

claude Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Claude finished @wezell's task in 3m 38s —— View job


Code Review

Reviewed the diff against origin/main. The changed set here is one production null-guard, a batch of integration-test hardening (swallowed-exception fixes, fixture-timing fixes, size parameterization), a new diagnostic script, and a Language.properties trim. Note: the changed files in this diff are not the CI shard-rebalancing described in the PR body — those .github/workflows / config.json / MainSuite changes are not present in origin/main...HEAD. This review covers what's actually in the diff.

New Issues

No issues found.

Notes (non-blocking)

  • AWSS3PublishingEndPoint.java:122-126 — The core fix is correct. PortalUtil.getUser() returns null when no request is bound to the thread (verified: PortalUtil.java:398-400 returns null for req==null), so the previous PortalUtil.getUser().getUserId() threw an NPE out of the catch block, masking the real S3 error. Guarding on null and relying on the already-logged Logger.warn is the right call. Import matches the return type.

  • The catch → throw new AssertionError(msg, e) conversions across ~10 test files (BrowserAPITest, ContentTypeTest, TailLogResourceTest, LocalTransactionAndCloseDBIfOpenedFactoryTest, etc.) are a clean, uniform improvement — they preserve the original message and the cause, which is exactly what assertTrue(msg, false) destroyed. No behavior regression: each still fails the test, now with a diagnosable stack trace.

  • RuleBundlerTest.java:48-78 — Good catch on the root cause. Moving fixture creation out of the @DataProvider (evaluated at suite construction) and into createRule() (called inside the test) genuinely fixes the "rule is null" ordering flake. The toString() override for readable parameterized names is a nice touch.

  • ImportUtilTest.java / Task240306...Test.java — Replacing fixed Thread.sleep(...) with Awaitility.await().atMost(...) and batching teardown destroys is sound. Two things worth a glance (not blocking):

  • find_swallowed.py:16import os, re, os, glob, collections imports os twice (harmless duplicate). Minor.

  • Language.properties — Removed keys (publishing-queue.bundles, .bundles.count, .column.items.view, .select-bundle.remove.confirm.header, publishing-queue.delete, .asset-list.remove-confirm.header) — this diff is backend/test-only, so I can't verify from here whether the corresponding frontend dm(...) / translate references were all removed. Assumption: the paired frontend PR drops these usages. What to verify: no remaining core-web reference to the deleted keys, or they'll render as raw key strings in the UI.

Overall: clean, well-reasoned changes with unusually good explanatory comments. Nothing blocking.
· issue-36940-ci-speedup

@github-actions github-actions Bot added Area : Backend PR changes Java/Maven backend code Area : CI/CD PR changes GitHub Actions/workflows Area : Documentation PR changes documentation files labels Aug 7, 2026
…with awaitility

Two of the slowest integration tests, both spending their time waiting rather
than testing.

Task240306MigrateLegacyLanguageVariablesTest -- 7.26m, the single most
expensive IT class (5.6% of all integration test time). The cost is not
executeUpgrade: testDataTaskIdempotency runs the upgrade twice and costs the
same as testExecuteUpgrade running it once. It is the teardown. cleanup()
walked the migration summary and did a find + destroy per inode, and a run
produces well over a hundred Language Variables (en=38, fr-fr=23, es-es=34,
plus others). destroy() is @WrapInTransaction, so that was ~150 transactions,
each with its own ES delete and cache invalidation, three times over.

Now batched through findContentlets(List) and destroy(List) -- one query and
one transaction. Teardown must not leave content behind for the next test, so
a batch failure falls back to the original per-contentlet loop, which tolerates
individual failures. removeExistingLanguageVariables() got the same treatment.
No test semantics change; only how the fixtures are torn down.

ImportUtilTest -- 34s of literal Thread.sleep, the most in the suite:

  30s  polling for content to appear in the index. It slept 30s BEFORE the
       first check, so it always paid the full cost even when the index was
       already caught up, and its retry bound (100 iterations) allowed a
       50-minute worst case.
   2s  waiting for an async import to land three contentlets
   1s  x2 waiting after addContentToIndex for content to become searchable

All four replaced with awaitility, following the pattern already used in
SiteSearchJobImplTest and other ITs: poll every 200ms, fail at 30s. The waits
now cost what they actually need rather than a fixed guess, and a genuinely
stuck index fails fast with a clear timeout instead of hanging.

Also leaves a note-worthy bug untouched, deliberately: removeLanguageAndContent
returns false immediately after destroying its first contentlet, so it never
reaches deleteLanguage. It is used by one test whose assertions depend on the
current return value, so fixing it is a behaviour change and belongs in its
own PR.

Not run locally (needs the full IT stack); verified by test-compile and by the
integration suites in this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH4Pes5S9d1AQorJBNeFJU
…way round

BrowserAPITest was the third most expensive integration test at 4.06m. Its
pagination tests picked a page size and then built a dataset several times
larger to cross the boundaries - 25 folders + 30 file assets to test "page 1
holds every folder plus one contentlet", and so on. Every FileAssetDataGen
writes a temp file, persists a contentlet and indexes it with WAIT_FOR, so the
dataset is nearly all of the runtime.

None of those assertions depend on absolute numbers. They depend on
relationships: page size vs folder count, and how much content remains after
the cursor. Scaling both sides down preserves every invariant exactly:

  page 1 fills with folders then tops up   25f + 30c -> 5f + 6c
  later page, fewer than a page remain     10f + 25c -> 2f + 5c
  mid-stream page, more remains            15f + 50c -> 3f + 10c
  exhaustive scan across permission gaps   20c       -> 10c
  scan limit stops the loop                20c       -> 5c

That is 145 file assets down to 36. The sizes are now named constants derived
from the page size, so the relationship under test is visible and the next
person has no reason to re-inflate the dataset. Three tests are renamed: their
old names hardcoded the counts and would have become lies.

FolderResourceTest got the same treatment, but only where it was safe. Three
of its five large loops are load-bearing and are left alone:

  moreThan20Folders (25)  regression test against a former hardcoded 20 cap
  defaultLimit40    (45)  pins the @DefaultValue("40") on the limit param
  limitMinusOne     (50)  proves limit=-1 bypasses that same 40 cap

Shrinking any of those below its cap would make the test pass trivially and
silently stop testing anything. Only withCustomLimit and withOffset - which
pin no production constant - are reduced, 30 subfolders each to 10.

Not run locally (needs the full IT stack); verified by test-compile and by the
integration suites in this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH4Pes5S9d1AQorJBNeFJU
wezell and others added 4 commits August 7, 2026 11:58
…rder-dependent tests

Re-sharding the integration suites surfaced two tests that only passed because of
where they sat in the run order. Both are genuine defects, so they are fixed
rather than pinned back into place.

AWSS3PublishingEndPoint - a NullPointerException thrown from inside a catch
block. validatePublishingEndPoint() catches S3 connection failures and reports
them to the user, but built that notification with:

    PortalUtil.getUser().getUserId()

PortalUtil.getUser() returns null whenever no request is bound to the thread:

    final HttpServletRequest req = HttpServletRequestThreadLocal.INSTANCE.getRequest();
    return req == null ? null : getUser(req);

So for any non-request caller - scheduled jobs, background tasks, tests - the
error handler NPE'd, replacing the S3 error it was trying to report with a
confusing NullPointerException and defeating the catch entirely. Now null-checked;
the notification is a UI concern, so with no user to notify the logged warning
above it is the whole story.

PublishingEndPointTest depended on that NPE not happening, which in turn depended
on some earlier test in the same suite leaving an HttpServletRequestThreadLocal
behind. In MainSuite1a something did; scheduled elsewhere, nothing does. The
tests also swallowed the exception:

    catch (Exception e) { Assert.assertTrue("No Exception should be thrown", false); }

which discards the cause - four retries produced four identical, contentless
messages and told us nothing. They now let the exception surface, so a failure
names itself. Renamed from *_returnException to *_returnsWithoutThrowing, since
they assert the opposite of what they claimed, and dropped the dead
`exceptionCatched` locals. The third test in the file had the identical pattern
and is fixed too - it has not failed yet, but only because of where it runs.

RuleBundlerTest created its Rules in a static @dataProvider. Providers are
evaluated when the suite is CONSTRUCTED - MainBaseSuite builds every runner up
front - so those Rules existed long before the test ran, and the ~65 test classes
scheduled in between could destroy them, giving "Cannot invoke Rule.getGroups()
because rule is null". The provider now returns only a flag and the Rule is
created inside the test, immediately before use. TestCase also gained a toString
so failures read "rule attached to a page" instead of "TestCase@5c2004ac".

Not run locally (needs the full IT stack); both modules test-compile clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH4Pes5S9d1AQorJBNeFJU
… catch block

A scan of all 845 integration test files found 18 catch blocks that fail the
test while discarding the exception that caused it. Three were fixed in the
previous commit after they cost a full CI run to diagnose; this fixes the
remaining 15.

The shape:

    catch (Exception e) { Assert.fail("Should work"); }

The test fails, but nothing about the actual cause reaches the log. Surefire
retries produce N identical, contentless messages - which is exactly what
happened with PublishingEndPointTest: four retries, four copies of "No Exception
should be thrown", and no way to tell what threw without reproducing locally.
Two of these were worse still - a bare `Assert.fail()` with no message at all
(FiltersTest:709, Task220413IncreasePublishedPushedAssetIdColTest:63).

All 15 now use:

    catch (Exception e) { throw new AssertionError("Should work", e); }

which fails the test identically but chains the cause, so the stack trace
survives into the CI log. Existing messages are preserved verbatim; the two
message-less sites get "Unexpected exception". Where the catch did other work
first - TailLogResourceTest re-interrupting the thread - that work is kept.

This is timely rather than incidental: 14 of the 15 sit in classes this PR moves
to a different shard. If any of them trips on a latent order dependency, it will
now say why instead of sending the next person on the same archaeology.

For balance, the same scan found 90 catch blocks that already pass the exception
along, so the codebase mostly gets this right - these were the outliers. It also
counted 163 empty catch blocks, but only 8 are the legitimate
`try { x; fail(); } catch (Expected e) {}` idiom; the rest are a mix of
deliberate best-effort cleanup and genuine hiding that a regex cannot separate.
Left alone rather than guessed at.

Not run locally (needs the full IT stack); test-compiles clean, and a re-scan
reports zero remaining.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH4Pes5S9d1AQorJBNeFJU
One conflict: MainSuite3a.java. This branch rewrites every MainSuite to use
fully-qualified class names and redistributes all 576 integration test classes
across 7 shards by measured time, while main added CacheResourceIntegrationTest
to MainSuite3a in #36917.

Resolved by keeping the repacked suites and re-adding the new class, so it is
still scheduled. Verified the merged suites contain exactly the same set of test
classes as origin/main - 576 on both sides, none dropped, none duplicated across
shards - and that both :dotcms-core and :dotcms-integration compile.

This kind of conflict is expected while this branch is open: any PR that adds a
test class touches a MainSuite file. The resolution is always the same - keep the
repacked shards, re-add the new class to whichever shard is lightest.
…ependencies

REVERT BEFORE MERGE - the restore value is in the comment above the setting.

Re-sharding the integration suites exposes tests that only passed because of
which suite-mates ran before them. Three found so far, each in a different
shard, and each cost a full ~40 minute run to find because fast-fail cancels
the other 25 jobs at the first failure.

Turning it off for this branch converts that into a single run that reports
every affected shard at once, so the remaining coupling can be fixed in one
batch instead of one per CI cycle.
…sk240306 test

testDropThenRecreateLanguageVariableContentType deleted the Language Variable
content type through ContentTypeAPI, then asserted that checkContentType()
recreated it. Removed, along with the now-orphaned
removeLanguageVariableContentType() helper and two imports it was the only user
of.

That delete should not be possible in the first place. The Language Variable
content type underpins all i18n, and losing it makes LanguageVariableAPIImpl
throw NotFoundInDbException and language resolution fall back to emitting raw
keys site-wide. #36958 tracks marking it `system` so ContentTypeFactoryImpl's

    if (type.system()) throw new DotDataException(...)

guard refuses the delete. Once that lands this test cannot work as written, and
until then it is a test deliberately exercising the destructive path we are
trying to close off - and it does so with DELETE_CONTENT_TYPE_ASYNC forced to
false, so the type really is gone for whatever runs next in that suite.

Nothing else is lost: its remaining assertions (executeUpgrade succeeds, summary
present, no failures) duplicate testExecuteUpgrade. What IS lost is coverage of
checkContentType() recreating the type when genuinely absent - a real situation
for legacy installs, which is why the upgrade task performs that check at all. A
comment at the removal site records this, and #36958 tracks re-covering it by
seeding the missing state directly rather than by calling the delete API.

Two side benefits:

- This was one of three ~2.4m tests in the single most expensive integration
  test class (7.26m, 5.6% of all IT test time), so it takes roughly a third off
  it.
- A grep of all 845 integration test files now finds zero API-level deletes of
  this content type: 24 references remain and every one is a read.

Not run locally (needs the full IT stack); test-compiles clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH4Pes5S9d1AQorJBNeFJU
…nt fixes

The rebalance works — measured on run 31203516115, with fail-fast disabled so
every shard reported:

    IT shards    22.2 - 26.2m   (was 38.9m)
    Postman      18.6 - 28.8m   (was 37.8m)

But collecting it costs more than this PR can carry. 3 of 7 integration shards
and 5 of 9 Postman shards failed, all from tests that were only passing because
of which suite-mates ran before them.

The Postman breakage was the bigger misjudgement on my part: those group names
were not labels, they were dependency clusters. `category-content` grouped
Category + ContentResourceV1 + Content_Resource because those collections share
data. Rebalancing purely on measured time shattered every one. The GraphQL
folder split broke the same way - "Page API - Testing 'page' field with inline
fragments" needs setup performed by the "Page API" folder, which landed in the
other shard.

Reverted here: test-matrix.yml shard lists, the seven repacked MainSuite files,
MainSuite3b/4a, dotcms-postman/config.json, index.js folder support, and the
temporary fail-fast:false.

Kept, because none of it depends on the rebalance:

  - the dead build-classes artifact (0.5m off the serial prefix, consumed by
    nothing)
  - AWSS3PublishingEndPoint: NullPointerException thrown from inside a catch
    block whenever no request is bound to the thread
  - 15 catch blocks that failed a test while discarding the cause
  - Task240306 teardown batched (~150 transactions -> 1) and its Language
    Variable content type delete removed (see #36958)
  - ImportUtilTest: 34s of Thread.sleep -> awaitility
  - pagination fixtures sized to the page size rather than the reverse

The full rebalance is preserved on `issue-36942-shard-rebalance-wip` at
f612b27 so it can be resumed rather than rebuilt. It is worth resuming: it
is an effective detector of tests that depend on their neighbours, and every
failure it surfaced so far has been a real defect - including a production NPE
and a deletable Language Variable content type. That is a test-independence
programme though, not a prerequisite for the fixes above.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH4Pes5S9d1AQorJBNeFJU
@wezell wezell changed the title perf(ci): rebalance IT and Postman shards on measured time (#36942) fix(ci): drop dead build artifact, guard an NPE in a catch block, and make integration tests report their failures Aug 7, 2026
wezell added a commit that referenced this pull request Aug 7, 2026
…#36961)

Closes part of #36947. **Stacked on #36960** — review that one first;
this PR's diff is against it.

## What

Gives the Apache Maven Build Cache Extension somewhere to keep its
results: a shared S3 bucket on OVH object storage.

The extension was **already installed and enabled**
(`.mvn/extensions.xml`, v1.2.0) — but local-only, so on a fresh CI
runner it cached nothing. The only missing piece was shared storage.

Scoped to the **Initial Artifact Build**, the serial prefix that gates
every test job. Test phases are deliberately *not* cached: a memoised
green run would mean "we did not run", and this suite has measured
flakes plus a hang that has burned a 122m job timeout.

## Transport

The extension speaks HTTP `PUT`/`GET`/`HEAD`; S3 needs SigV4. Rather
than add a Maven S3 wagon — the available ones aren't maintained
(`seahen` 1.3.3 is 2021, `gkatzioura` 2.3 is 2019, both AWS SDK v1) —
[`aws-sigv4-proxy`](https://github.com/awslabs/aws-sigv4-proxy) runs as
a container and signs on the way out, and Maven talks to `127.0.0.1`.
**No new Maven dependency.**

## Security

A build cache untrusted code can write is a supply-chain vector: a
poisoned entry is replayed as a build output on a trusted ref. This is
[CVE-2025-36852](https://www.cve.org/CVERecord?id=CVE-2025-36852)
("CREEP"), which killed Nx's `@nx/s3-cache` and its siblings.

**The control is the credential, not the client** — a job holding a
writable key can bypass Maven entirely with one `aws s3 cp`, so
`remote.save.enabled` is defence in depth, not the boundary.

| Ref | Key | Writes |
|---|---|---|
| PR | `..._ACCESS_KEY_RO` (GetObject only) | no |
| merge queue / trunk | `..._ACCESS_KEY` | yes |
| fork PR | none — builds uncached | no |

Plus:
- `remote.save.final=true` — an existing entry is never overwritten.
- The action **asserts** its read-only key is read-only (one `PUT`,
expects `403`) instead of assuming it. A writable "read-only" key looks
identical to a correct setup until abused.
- Writing builds record a `provenance.json` beside each entry (ref, sha,
run id, actor), first-writer-wins — nothing else in a bucket says which
ref produced a hash.

## The subtle one: `alwaysRunPlugins`

Load-bearing, not tuning. On a cache hit the extension skips cached
plugin executions **including `install:install`** — measured: 1 jar in
`~/.m2/repository` after a cold build, **0 after a hit**. This job
exists to publish that repository as the `maven-repo` artifact ~25 test
jobs consume. Same story for `docker-maven-plugin:build`, which writes
the `docker-build.tar` the next step uploads.

Both would have looked perfectly green on the cold populate run and
broken everything on the first *warm* one.

## Verification

Against MinIO before any of this was wired:

| Check | Result |
|---|---|
| `PUT` / `GET` / `HEAD` through the proxy | `200` / `200` / `200` |
| Missing key | `404` (a `403` reads as a hard error, not a miss) |
| 10 MB body round-trip | byte-identical |
| Build with an **empty** local cache | `Found cached build, restoring …
by checksum` |
| Remote unreachable | build exits `0`, logs an error, rebuilds |

And separately, because `cicd_comp_build-phase.yml` fails a PR on a
dirty tree while `openapi.yaml` is a tracked file generated at compile:
built `:dotcms-core --am` twice — 12 modules restored including
`dotcms-core`, `openapi.yaml` md5 identical, `git status` unchanged.

## Releases are not affected

Release, LTS, nightly, manual-deploy and CLI-release workflows don't
pass the secrets, and **no workflow in this repo uses `secrets:
inherit`** (verified). The action sees empty credentials and exports an
empty `BUILD_CACHE_ARGS`; those pipelines build from scratch exactly as
today. A release is the build where "we didn't actually compile this" is
least acceptable and the saving is worth least.

## Turning it off

| Scope | How |
|---|---|
| One PR | label `CI: no build cache` |
| Everything, now | repo/org variable `BUILD_CACHE_DISABLED=true` |
| Local build | `-Dmaven.build.cache.enabled=false` |
| Force rebuild, still publish | `-Dmaven.build.cache.skipCache=true` |

## Expected effect, honestly

Ceiling is the ~7.0m of Maven time inside a 14.2m build job, against a
74–103m PR wall clock. Real but modest — the twin-tail shard rebalance
(#36943) is still the bigger lever. `dotcms-core-web` only participates
once #36960 lands.

**The first merge-queue run is the canary**: it's what proves the OVH
SigV4 handshake and the region derived from the endpoint host. If either
is wrong the cache disables itself with a warning rather than failing
the build.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_014a2iJy9JXRBSVdKBbmoZ2S

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area : Backend PR changes Java/Maven backend code Area : CI/CD PR changes GitHub Actions/workflows Area : Documentation PR changes documentation files

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

CI: rebalance integration and Postman shards to cut PR wall clock

1 participant