initrd/bin/kexec-sign-config.sh: stop re-hashing stale default entry paths on -u update#2150
Open
tlaurion wants to merge 1 commit into
Open
initrd/bin/kexec-sign-config.sh: stop re-hashing stale default entry paths on -u update#2150tlaurion wants to merge 1 commit into
tlaurion wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Collaborator
Author
|
QubesOS automated testing was testing (results still available as writing)
Note that replacing xen-4.19.4.gz with xen-4.19.5.gz landed July 2nd, as per normal testing->release falling through all current automated testing in place for QubesOS QA/Dasharo. CC: @marmarek @macpijan @pietrushnic, let me know how we want to handle this forward.
|
8fe881f to
7acbb08
Compare
…paths on -u update Remove the -u block that re-hashed file paths extracted from the old kexec_default_hashes.txt. Those paths are from a previously saved default boot entry. If a distro replaced a versioned boot file in-place (i.e. atomically deletes old binaries while providing new ones), the old path no longer exists on disk and sha256sum fails, aborting the entire update with DIE. Qubes OS is the first distro that replaces boot binaries atomically without fan-out: a dom0 update replaces xen-4.19.4.gz with xen-4.19.5.gz in-place (QubesOS/qubes-vmm-xen#220, shipped to stable via QubesOS/updates-status#6746 on 2026-07-02). Other distros (Fedora, Arch, Debian, NixOS, Ubuntu) keep old kernels alongside new ones, leaving deletion to an OS cleanup policy, so the old paths still resolve and the re-hash failure was never observed there. The re-hash had been present since its introduction in 2018 but went unnoticed because no OS had ever removed boot binaries atomically without leaving the old files in place — Qubes OS automated testing on heads commit 426750a (2026-04-22, pre_release_dasharo_101_rc3) passed because xen 4.19.4 was still current and the old paths resolved (openQA tests T430 openqa.qubes-os.org/tests/187218, V560TU openqa.qubes-os.org/tests/186420). It became blocking only after xen 4.19.5 replaced 4.19.4 on disk, at which point any user with a default boot entry pointing at xen-4.19.4.gz would hit the fatal failure. The failure is now fatal because the -u (update) and -r (rollback counter) flags are passed together in a single call. When the re-hash fails, DIE is called before the rollback counter is ever incremented, leaving the system with a partially updated kexec_hashes.txt but no counter bump. This coupling was introduced in commit 8bfae52 (Oct 2020), which restructured update_checksums to always call kexec-sign-config with -u hardcoded and -r as extparam. Regression lineage: - Mar 14, 2018 - Kyle Rankin introduced the re-hash in update_checksums() inside gui-init (commit 769f6a7). - Jul 5, 2019 - Matt DeVillier moved update_checksums() to /etc/functions (commit ed2f19d). - Oct 8, 2020 - alex-nitrokey moved the re-hash block into kexec-sign-config under the -u flag and coupled it with -r (rollback) in the same call (commit 8bfae52). - Through 2026 - No OS had removed boot binaries atomically without leaving old files in place, so the coupled failure went untriggered. Mitigation on unpatched firmware (recovery shell): mount -o rw,remount /boot rm /boot/kexec_default_hashes.txt mount -o ro,remount /boot reboot.sh Then re-run Update checksums. The -e guard skips the re-hash when the file is gone. Then set a new default from the boot menu. Fix: skip the re-hash entirely. The user re-saves the default boot entry from the boot menu to regenerate kexec_default_hashes.txt. Signed-off-by: Thierry Laurion <insurgo@riseup.net>
7acbb08 to
3950b73
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove the -u block that re-hashed file paths extracted from the old kexec_default_hashes.txt. Those paths are from a previously saved default boot entry. If a distro replaced a versioned boot file in-place (e.g. Qubes OS bumping xen-4.19.4.gz to xen-4.19.5.gz), the old path no longer exists on disk and sha256sum fails, aborting the entire update with DIE.
Read: delete old binaries at the same time of providing new ones, which never happened before.
Qubes OS does NOT fan out old and new xen binaries on /boot: a dom0 update replaces xen-4.19.4.gz with xen-4.19.5.gz in-place (QubesOS/qubes-vmm-xen#220, shipped to stable via QubesOS/updates-status#6746 on 2026-07-02). Other distros (Fedora, Arch, Debian, NixOS, Ubuntu...) keep old kernels+initramfs alongside new ones, so the old paths still resolve and the re-hash failure was never observed there.
The re-hash had been present since its introduction in 2018 but went unnoticed because most distros keep old kernel/xen files on disk alongside new ones, so the old paths still resolved.
The failure is now fatal because the -u (update) and -r (rollback counter) flags are passed together in a single call. When the re-hash fails, DIE is called before the rollback counter is ever incremented, leaving the system with a partially updated kexec_hashes.txt but no counter bump. This coupling was introduced in commit 8bfae52 (Oct 2020), which restructured update_checksums to always call kexec-sign-config with -u hardcoded and -r as extparam.
Regression lineage:
Mitigation on unpatched firmware (recovery shell):
Then re-run Update checksums. The -e guard skips the re-hash when the file is gone. Then set a new default from the boot menu.
Fix: skip the re-hash entirely. The user re-saves the default boot entry from the boot menu to regenerate kexec_default_hashes.txt.
Was discussed over matrix's Dasharo Premier support channel (subscribers only) here.
Affects all Heads + QubesOS 4.3 users