Build speed improvements

[SJrX]

Release/CI builds are slow and highly variable (observed 7m–24m). A Develocity build scan of a representative ~7m43s run (https://gradle.com/s/hllckx5e6jnw2) shows where the time goes and what's worth fixing.

Findings from the build scan

• ~49% of the build is configuration, not compilation/test:
  • Initialization & configuration: 3m49s; Execution: 3m54s.
  • Model configuration 1m40s (IntelliJ Platform plugin building its classpath model) + Configuration resolution 1m18s.
• The ~948 MiB IntelliJ IDE is re-downloaded every build (ideaIC-2024.2-aarch64.tar.gz, 1 GiB of network total). The build-environment image already warms the Gradle cache, but the baked copy isn't used at runtime — almost certainly because GRADLE_USER_HOME/HOME at runtime differs from where the warm-up wrote it.
• Remote build cache works well (85% hit, saved ~1m20s) — task outputs are cached; dependencies and configuration are not.
• Memory is not the bottleneck for the JVM (heap capped at 512 MiB, GC ~5s). The :test swings (1m21s → 6m37s) track CPU contention from co-scheduled pods, not RAM.
• Two Task.project at execution time deprecations (config-cache blockers).

Planned in the first PR

• Pin GRADLE_USER_HOME in ci/Build-Environment.Dockerfile so the already-baked IDE/dependency cache is actually used at runtime (hermetic: per-pod COW from the immutable image, distributed via the registry — no shared mutable host state). Expected ~1–1.5 min off every build.
• Bump the build-environment image to Ubuntu 24.04 (handle the UID 1000 ubuntu default-user collision).
• Fix the Task.project at execution time deprecations (build warning cleanup; also a prerequisite if configuration cache is ever pursued).

Considered, deferred (want more data first)

• Node anti-affinity / CPU requests on release pods to spread builds across nodes and stop :test contention. Holding until we capture a scan from a slow run to confirm the contention hypothesis.
• Configuration cache to reclaim the 1m40s model-configuration. Needs the deprecations fixed and cross-build persistence + a stable checkout path, which conflicts with keeping builds hermetic — not worth it until measured.

References

• Representative scan: https://gradle.com/s/hllckx5e6jnw2

[SJrX]

Correction on the deferred scheduling item: the lever is inter-pod anti-affinity, not node anti-affinity.

Goal: when multiple builds run concurrently, spread their (CPU-heavy) pods across nodes instead of letting the scheduler pack them onto one Raspberry Pi CM5 (4 cores), which is the suspected cause of the :test time swings.

Approach when we act on it:

• podAntiAffinity / preferredDuringSchedulingIgnoredDuringExecution (soft — must still schedule when concurrent builds > nodes).
• topologyKey: kubernetes.io/hostname.
• labelSelector on a stable label shared across build pods. Note jenkins/label is per-build so it won't match siblings; use maintainer: jenkins or add a dedicated label (e.g. workload: gradle-build) to the Build-stage worker-pod so the rule targets only the heavy pods.

Still gated on capturing a build scan from a slow/contended run to confirm CPU contention before changing scheduling.

[SJrX]

Result (PR #462, warm-cache replay)

Validated on a warm-cache run (PR-462 build #3, scan https://gradle.com/s/eitr2s3rjkjy2):

	Baseline	Warm (after #462)
Total	7m43s	4m40s (~40% faster)
Init & configuration	3m49s	1m02s
Network downloaded	1 GiB	baked (IDE no longer re-downloaded)
Build-cache hit rate	85%	85%
Build deprecations	2	0

The first post-#462 build had a cold cache (1% hit, 6m11s) purely because it was the first build on the new Ubuntu 24.04 image with changed build logic; the cache recovers to 85% on the next run, as expected.

Remaining floor (largely irreducible)

• :test ~1m48s — needs Develocity Test Distribution / Predictive Test Selection (paid) to shrink.
• :compileTestKotlin ~1m4s — never caches because its classpath includes the per-build version-stamped plugin jar.

Remaining levers — low priority at 4m40s

• Repository content filtering: trims config resolution, but config is already ~1 min.
• Inter-pod anti-affinity: only if a busy-period scan shows contention.
• Configuration cache: biggest theoretical win, but conflicts with keeping builds hermetic.

Suggest closing this once #462 merges, leaving the above as parked/conditional follow-ups.