Skip to content

feat(freshness): family-aware coverage so sweep-validated kernels count#93

Merged
ErenAri merged 1 commit into
mainfrom
feat/family-aware-freshness
Jul 21, 2026
Merged

feat(freshness): family-aware coverage so sweep-validated kernels count#93
ErenAri merged 1 commit into
mainfrom
feat/family-aware-freshness

Conversation

@ErenAri

@ErenAri ErenAri commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Closes #92.

Problem

A stock cloud image can never ship a kernel newer than itself, while kernel-crawler tracks the newest package. So image-based profiles were flagged stale permanently, no matter how much validation existed.

After #91 we literally had ubuntu-22.04-5.15-k5.15.0-186 proving 5.15.0-186 loads and attaches on jammy — while ubuntu-22.04-5.15 still reported stale. The oracle was asking "did this exact profile boot the newest kernel?" instead of the useful question, "has the newest kernel for this series been validated at all?"

Change

Adds a covered status. When a kernel-sweep profile in the same family (<base>-k<release>) has validated a kernel at least as new as the newest shipping one, the base profile reports covered, names the profile that proves it (covered_by), and is excluded from StaleCount.

Family matching keys off the full <base>-k prefix and requires a digit after it, which gives two useful properties:

  • Sibling variants stay independentubuntu-22.04-5.15-lockdown is only covered by its own …-lockdown-k… derivatives, never by plain ubuntu-22.04-5.15 ones (verified in tests and on real data).
  • Ids that merely contain -k (e.g. a -kvm-… suffix) cannot be mistaken for derivatives.

Effect on the committed baselines

Stale 11 → 6, with 7 profiles correctly reattributed:

Profile covered_by
ubuntu-18.04-4.15 …-k4.15.0-213
ubuntu-20.04-5.4 …-k5.4.0-218
ubuntu-22.04-5.15 …-k5.15.0-186
ubuntu-22.04-5.15-lockdown …-lockdown-k5.15.0-186
ubuntu-24.04-6.8 …-k6.8.0-136
ubuntu-25.10-6.17 …-k6.17.0-41
ubuntu-25.10-minimal-6.17 …-k6.17.0-41

The signal is not weakened — the remaining 6 are genuinely unvalidated on their newest kernel: almalinux-9, rocky-9, centos-stream-9 (RHEL-family; kernel-sweep is Ubuntu-only), plus oracle-linux-9-uek7 and rhel-8 (not refreshable by image swap) and amazon-linux-2 (which went newly stale mid-session — Amazon shipped 5.10.260 after we validated 5.10.259, a good demonstration that the oracle still bites).

Verification

New TestEvaluateFamilyCoverage covers: base covered by its derivative; sibling variant not covered by the base's derivative; a derivative that only reached an older kernel not covering; and the digit guard. Full suite, gofmt, and golangci-lint all clean. Docs updated in docs/image-pipeline.md; the weekly lane's Markdown report now explains the covered count.

A stock cloud image can never ship a kernel newer than itself, while
kernel-crawler tracks the newest package, so image-based profiles were
flagged stale permanently no matter how much validation existed. After
#91 we had ubuntu-22.04-5.15-k5.15.0-186 proving 5.15.0-186 loads on
jammy, yet ubuntu-22.04-5.15 still reported stale -- the oracle was
asking 'did this exact profile boot the newest kernel?' instead of 'has
the newest kernel for this series been validated at all?'.

Add a 'covered' status: when a kernel-sweep profile in the same family
(<base>-k<release>) has validated a kernel at least as new as the
newest shipping one, the base profile reports covered, names the
profile that proves it (covered_by), and is excluded from StaleCount.

Family matching keys off the full '<base>-k' prefix and requires a
digit after it, so sibling variants stay independent
(ubuntu-22.04-5.15-lockdown is only covered by its own -lockdown-k
derivatives) and ids that merely contain '-k' cannot match.

Against the committed baselines this takes stale 11 -> 6 without
weakening the signal: the remaining 6 are genuinely unvalidated on
their newest kernel (RHEL-family, which kernel-sweep does not support,
plus oracle-linux-9-uek7 and rhel-8).

Closes #92.

Signed-off-by: ErenAri <erenari27@gmail.com>
Copilot AI review requested due to automatic review settings July 21, 2026 10:46

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@ErenAri
ErenAri merged commit 48781ca into main Jul 21, 2026
8 of 9 checks passed
@ErenAri
ErenAri deleted the feat/family-aware-freshness branch July 21, 2026 10:48
ErenAri added a commit that referenced this pull request Jul 21, 2026
* feat(kernel-sweep): support the RHEL family (dnf/grubby)

kernel-sweep was Ubuntu-only, so the RHEL-family profiles could never be
credited by the family-aware freshness added in #93 -- there was no way
to validate their newest kernel at all.

Extend it to AlmaLinux, Rocky and CentOS Stream:

- RHELKernelRPMs derives the bootable kernel RPMs from the crawler
  entry. The crawler publishes only kernel-devel, from AppStream, while
  kernel-core and the modules packages live in BaseOS; swapping the
  component and file name inherits mirror, repository version, arch and
  any per-letter subdirectory (Rocky's Packages/k/).
- In-guest install is now family-aware: dnf plus 'grubby --set-default'
  (which selects by vmlinuz path, with none of the grub menu-title
  matching Debian needs).
- KernelInstallFamily centralises the distro -> family mapping used by
  validation, install and sweep. Oracle (UEK ships from its own repos),
  Amazon, and the immutable image-based systems stay unsupported.
- Profile validation accepts .rpm for RHEL-family profiles and still
  rejects it for Debian ones.

Also fixes a latent bug: --target defaulted to ubuntu-generic, which is
wrong for the -minimal profiles (they run ubuntu-kvm). The crawler
mapping now comes from vm/kernel-baselines.yaml for the base profile --
including release_contains, which keeps el9 and el10 apart under one
distro key -- with flags still overriding.

Validated on real boots: almalinux-9, rocky-9 and centos-stream-9 each
installed and booted their newest kernel (5.14.0-687.26.1.el9_8,
5.14.0-687.26.1.el9_8, 5.14.0-725.el9) and passed. Freshness stale
count 6 -> 3; the remainder are amazon-linux-2, oracle-linux-9-uek7 and
rhel-8, none reachable by a base-image or sweep refresh.

Signed-off-by: ErenAri <erenari27@gmail.com>

* fix(kernel-sweep): https package URLs and real signature verification

Hardens the kernel install path flagged by security review.

Transport: kernel-crawler publishes plain http mirror URLs, and the
derivations passed the scheme through, so a kernel installed into the
guest was modifiable in transit. Both derivations now emit https (every
mirror serves the same paths over TLS, verified), and profile validation
rejects any kernel_packages entry that is not https, so a hand-written
or future profile cannot reintroduce plaintext. All previously generated
sweep profiles are regenerated accordingly.

Verification: the RHEL install used --nogpgcheck, disabling package
signature checks entirely. It now runs 'rpm --checksig' and installs
with localpkg_gpgcheck=1 -- dnf skips GPG verification for local package
files unless that option is set, so dropping --nogpgcheck alone would
have verified nothing.

Enforcing verification exposed a second problem in real boots: AlmaLinux
pre-imports the distro keys into the rpm keyring, but Rocky and CentOS
Stream cloud images only place them on disk, so both failed with
'SIGNATURES NOT OK' for valid vendor packages. The install now imports
the keys shipped in the image before verifying -- the same trust anchor
dnf uses for the image's configured repositories.

Re-validated on real boots after the change: almalinux-9, rocky-9,
centos-stream-9 (RHEL, signatures verified) and ubuntu-24.04 (https
through the dpkg path) all pass.

Note: the Debian path gets transport integrity only. dpkg cannot verify
signatures on standalone .deb files; apt verifies at the index level
instead.

Signed-off-by: ErenAri <erenari27@gmail.com>

---------

Signed-off-by: ErenAri <erenari27@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

freshness: make the oracle family-aware so sweep-validated kernels count for their base profile

2 participants