Kernel Panic when booting linux6.18.14_1

[TeusLollo]

Is this a new report?

Yes

System Info

Void 6.18.13_1 x86_64 AuthenticAMD uptodate rrrrm

(Also happening on a separate Intel-based platform)

Package(s) Affected

linux6.18-6.18.14_1 linux6.18-headers-6.18.14_1

Does a report exist for this bug with the project's home (upstream) and/or another distro?

No response

Expected behaviour

The kernel should initialize upon booting and bring the system out of stage 1

Actual behaviour

Simply put, testing on two separate boxes (An AMD-based one, and an Intel-based one) that have been both
running linux6.18-6.18.13_1 with no issues whatsoever, upon updating only linux6.18-6.18.14_1 linux6.18-headers-6.18.14_1, a kernel panic results.

System stuck in stage 1, attempting to send signals yields the familiar output "warning: signals only work in stage 2", thus requiring an hard reboot through the firmware button

Given how tests were applied upon two radically-different platforms (An AM5 AMD one, and an Intel-based notebook one), I'm ruling out vendor-specific hardware issues

Despite having both socklog-unix and nanoklogd enabled as runit services, it was impossible to isolate a satisfying error log.
Attached is thus a manual camera capture of the kernel panic at hand when booting upon the Intel-based platform

Steps to reproduce

1 Install linux6.18-6.18.14_1 linux6.18-headers-6.18.14_1 (Comes with the territory that you probably have dkms installed if you are actually installing headers). dracut is utilized with default values (/etc/dracut.conf.d/ is empty, /etc/dracut.conf is untouched)
2 Reboot
3 Kernel panic with system stuck in stage 1, tested on two separate boxes with separate vendor platforms that have been running solidly with linux6.18-6.18.13_1
4 Performing an hard reboot, and rebooting with linux6.18-6.18.13_1 instead, fixes the issue, and platforms work fine thereafter

EDIT: Before anyone else asks, no, there are no free space constraints, and vkpurge is run once a week

EDIT: Possibly a regression: https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.18.14

[dogknowsnx]

Wild guess, does vkpurge ring a bell?
https://docs.voidlinux.org/config/kernel.html

[TeusLollo]

Wild guess, does vkpurge ring a bell? https://docs.voidlinux.org/config/kernel.html

I don't think I understand.
If I ran vkpurge now, I'd be left with a panicking kernel from the linux6.18-6.18.14_1 package.
At best, I'd need to downgrade first with xdowngrade, and then put the the linux6.18-* package on hold to prevent updates

If you meant that I did something funky with config, I already stated that dracut config was left as is.
dkms is installed and builds ABI-compatible kernel modules using each updated -headers package, which has been working fine since I've been on Void, basically, and has been working fine since I manually installed the linux6.18 package.
I went through each 14 of every linux6.18-6.18.*_* update with the same exact setup, and only got this kernel panic on updating to linux6.18-6.18.14_1 with linux6.18-headers-6.18.14_1

I don't expect a dramatic shift in functionality between two updates of the same LTS kernel, thus something may have happened here, especially considering I got this same exact result on two very different machines.

Care to elaborate more?

[dogknowsnx]

Are you using similar setups on both machines, e.g. separate boot partitions of equal size? If so, did you ever clean old kernels?

[TeusLollo]

Are you using similar setups on both machines, e.g. separate boot partitions of equal size? If so, did you ever clean old kernels?

Setups are nearly identical, but machines are from two separate vendors, with largely different driver configurations at boot, given different internal components. as generated by dracut (Which is run on an obsolutely default config with no modifications whatsoever, and has been running perfectly through all 14 linux6.18-6.18.*_* updates since it was installed)

Kernels (Along with dracut-generated config-* files, are deposited into /boot, not /boot/efi, since grub allows booting kernels that are in a different directory than /boot/efi, thus /boot has the same free space available as the entire / (Root) partition (Hundreds of GiB on both machines).
This is entirely dependent on the bootloader, not the kernel, and in every case such a change is not expected on an LTS kernel and would be announced beforehand
And, in every case, I clean obsolete kernels once a week

If you're suggesting troubles with efistub, then it wouldn't be finding a kernel to boot at all and would be throwing a very different error altogether.

EDIT: Looks like 24722e3 made some changes to file srcpkgs/linux6.18/files/x86_64-dotconfig‎, flag CONFIG_ATH9K_AHB=y was removed on x86_64 architectures, which references a component that is not present on any of my machines (https://wireless.docs.kernel.org/en/latest/en/users/drivers/ath9k.html), thus it shouldn't matter

[dogknowsnx]

I clean obsolete kernels once a week

You didn't mention that in the OP, hence my wild guess. Fwiw, linux6.18-6.18.14_1 is booting fine for me on two different machines. I can't help you, sorry. Good luck!

[TeusLollo]

I accidentally downgraded to only linux6.18-6.18.13_1, which meant the install hook rebuilt linux6.18-6.18.13_1 with linux6.18-6.18.14_1-headers, I rebooted, and it's actually similarly working fine, which means there are no significant ABI changes between linux6.18-6.18.13_1-headers and linux6.18-6.18.14_1-headers, hinting the problem is into linux6.18-6.18.14_1

I also attempted installing linux6.19-6.19.4-1, and got the following output:
linux6.19-6.19.4_1: verifying RSA signature... ERROR: linux6.19-6.19.4_1: the RSA signature is not valid! ERROR: linux6.19-6.19.4_1: removed pkg archive and its signature. ERROR: Transaction failed! see above for errors.

I'm using repo-fastly.voidlinux.org as repository, which shouldn't contain any mismatches given how it was committed 14 hours ago as of now

Sorry for pinging, @Duncaen, but I think something went funky with this update, I apologize in advance, but you should take a look

[Duncaen]

I accidentally downgraded to only linux6.18-6.18.13_1

That shouldn't cause any issues and if you didn't run vkpurge you probably still had both the kernel and the headers installed. If the headers are actually required for DKMS, then that should just fail. The directories are all versioned so its not possible to actually use the wrong headers.

Besides the signature issues, which should be fixed now but might need some time to arrive at all mirrors I don't think there is anything wrong with the packages itself.

The kernel changelog for this release is quite big so it may be a regression, specifically those changes in relation to netlink which seem relevant based on your backtrace. https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.18.14

[TeusLollo]

That shouldn't cause any issues and if you didn't run vkpurge you probably still had both the kernel and the headers installed. If the headers are actually required for DKMS, then that should just fail. The directories are all versioned so its not possible to actually use the wrong headers.

Besides the signature issues, which should be fixed now but might need some time to arrive at all mirrors I don't think there is anything wrong with the packages itself.

The kernel changelog for this release is quite big so it may be a regression, specifically those changes in relation to netlink which seem relevant based on your backtrace. https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.18.14

Indeed there are no issues and there were no failures upon install hook detected. I also already did a manual backup of linux6.18-6.18.13_1 and linux6.18-6.18.13_1-headers, downgraded, and put the packages on hold, so I should be good for a while

Thanks for the link, looks like they're already working on a fix. I will keep this open for now, since I guess this may affect other users, and will report back when 6.18.15 is commited.

[Duncaen]

Next version of 6.18 and 6.19 should fix the issue:
https://lore.kernel.org/all/bb9ab61c-3bed-4c3d-baf0-0bce4e142292@moonlit-rail.com/

[TeusLollo]

Installing linux6.18-6.18.15_1 and linux6.18-6.18.15_1-headers seems to have fixed kernel panics upon the AMD-based platform. Will let it test itself for a few hours, then retry also upon the Intel-based platform

If all goes well, I will be closing this in about 24 hours.

Anyone else is free to make their points on here in the meantime