feat(freshness): family-aware coverage so sweep-validated kernels count#93
Merged
Conversation
A stock cloud image can never ship a kernel newer than itself, while kernel-crawler tracks the newest package, so image-based profiles were flagged stale permanently no matter how much validation existed. After #91 we had ubuntu-22.04-5.15-k5.15.0-186 proving 5.15.0-186 loads on jammy, yet ubuntu-22.04-5.15 still reported stale -- the oracle was asking 'did this exact profile boot the newest kernel?' instead of 'has the newest kernel for this series been validated at all?'. Add a 'covered' status: when a kernel-sweep profile in the same family (<base>-k<release>) has validated a kernel at least as new as the newest shipping one, the base profile reports covered, names the profile that proves it (covered_by), and is excluded from StaleCount. Family matching keys off the full '<base>-k' prefix and requires a digit after it, so sibling variants stay independent (ubuntu-22.04-5.15-lockdown is only covered by its own -lockdown-k derivatives) and ids that merely contain '-k' cannot match. Against the committed baselines this takes stale 11 -> 6 without weakening the signal: the remaining 6 are genuinely unvalidated on their newest kernel (RHEL-family, which kernel-sweep does not support, plus oracle-linux-9-uek7 and rhel-8). Closes #92. Signed-off-by: ErenAri <erenari27@gmail.com>
ErenAri
added a commit
that referenced
this pull request
Jul 21, 2026
* feat(kernel-sweep): support the RHEL family (dnf/grubby) kernel-sweep was Ubuntu-only, so the RHEL-family profiles could never be credited by the family-aware freshness added in #93 -- there was no way to validate their newest kernel at all. Extend it to AlmaLinux, Rocky and CentOS Stream: - RHELKernelRPMs derives the bootable kernel RPMs from the crawler entry. The crawler publishes only kernel-devel, from AppStream, while kernel-core and the modules packages live in BaseOS; swapping the component and file name inherits mirror, repository version, arch and any per-letter subdirectory (Rocky's Packages/k/). - In-guest install is now family-aware: dnf plus 'grubby --set-default' (which selects by vmlinuz path, with none of the grub menu-title matching Debian needs). - KernelInstallFamily centralises the distro -> family mapping used by validation, install and sweep. Oracle (UEK ships from its own repos), Amazon, and the immutable image-based systems stay unsupported. - Profile validation accepts .rpm for RHEL-family profiles and still rejects it for Debian ones. Also fixes a latent bug: --target defaulted to ubuntu-generic, which is wrong for the -minimal profiles (they run ubuntu-kvm). The crawler mapping now comes from vm/kernel-baselines.yaml for the base profile -- including release_contains, which keeps el9 and el10 apart under one distro key -- with flags still overriding. Validated on real boots: almalinux-9, rocky-9 and centos-stream-9 each installed and booted their newest kernel (5.14.0-687.26.1.el9_8, 5.14.0-687.26.1.el9_8, 5.14.0-725.el9) and passed. Freshness stale count 6 -> 3; the remainder are amazon-linux-2, oracle-linux-9-uek7 and rhel-8, none reachable by a base-image or sweep refresh. Signed-off-by: ErenAri <erenari27@gmail.com> * fix(kernel-sweep): https package URLs and real signature verification Hardens the kernel install path flagged by security review. Transport: kernel-crawler publishes plain http mirror URLs, and the derivations passed the scheme through, so a kernel installed into the guest was modifiable in transit. Both derivations now emit https (every mirror serves the same paths over TLS, verified), and profile validation rejects any kernel_packages entry that is not https, so a hand-written or future profile cannot reintroduce plaintext. All previously generated sweep profiles are regenerated accordingly. Verification: the RHEL install used --nogpgcheck, disabling package signature checks entirely. It now runs 'rpm --checksig' and installs with localpkg_gpgcheck=1 -- dnf skips GPG verification for local package files unless that option is set, so dropping --nogpgcheck alone would have verified nothing. Enforcing verification exposed a second problem in real boots: AlmaLinux pre-imports the distro keys into the rpm keyring, but Rocky and CentOS Stream cloud images only place them on disk, so both failed with 'SIGNATURES NOT OK' for valid vendor packages. The install now imports the keys shipped in the image before verifying -- the same trust anchor dnf uses for the image's configured repositories. Re-validated on real boots after the change: almalinux-9, rocky-9, centos-stream-9 (RHEL, signatures verified) and ubuntu-24.04 (https through the dpkg path) all pass. Note: the Debian path gets transport integrity only. dpkg cannot verify signatures on standalone .deb files; apt verifies at the index level instead. Signed-off-by: ErenAri <erenari27@gmail.com> --------- Signed-off-by: ErenAri <erenari27@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #92.
Problem
A stock cloud image can never ship a kernel newer than itself, while kernel-crawler tracks the newest package. So image-based profiles were flagged
stalepermanently, no matter how much validation existed.After #91 we literally had
ubuntu-22.04-5.15-k5.15.0-186proving5.15.0-186loads and attaches on jammy — whileubuntu-22.04-5.15still reportedstale. The oracle was asking "did this exact profile boot the newest kernel?" instead of the useful question, "has the newest kernel for this series been validated at all?"Change
Adds a
coveredstatus. When a kernel-sweep profile in the same family (<base>-k<release>) has validated a kernel at least as new as the newest shipping one, the base profile reportscovered, names the profile that proves it (covered_by), and is excluded fromStaleCount.Family matching keys off the full
<base>-kprefix and requires a digit after it, which gives two useful properties:ubuntu-22.04-5.15-lockdownis only covered by its own…-lockdown-k…derivatives, never by plainubuntu-22.04-5.15ones (verified in tests and on real data).-k(e.g. a-kvm-…suffix) cannot be mistaken for derivatives.Effect on the committed baselines
Stale 11 → 6, with 7 profiles correctly reattributed:
The signal is not weakened — the remaining 6 are genuinely unvalidated on their newest kernel:
almalinux-9,rocky-9,centos-stream-9(RHEL-family; kernel-sweep is Ubuntu-only), plusoracle-linux-9-uek7andrhel-8(not refreshable by image swap) andamazon-linux-2(which went newly stale mid-session — Amazon shipped5.10.260after we validated5.10.259, a good demonstration that the oracle still bites).Verification
New
TestEvaluateFamilyCoveragecovers: base covered by its derivative; sibling variant not covered by the base's derivative; a derivative that only reached an older kernel not covering; and the digit guard. Full suite,gofmt, andgolangci-lintall clean. Docs updated indocs/image-pipeline.md; the weekly lane's Markdown report now explains thecoveredcount.