SNO provision from Hetzner rescue

WARNING: This repository and its scripts are neither supported nor endorsed by Red Hat or Hetzner. It is experimental and not intended for production use. Use it under your own risk.

These are some simple scripts meant to be run from Hetzner Rescue System to be able to install OpenShift from the assisted installer or using the agent-based installer.

The scripts are:

• hetzner-sno-provision-host.sh <iPXE script URL>: It is designed to run run from Hetzner Rescue System. You have to provide the iPXE URL generated by assisted installer in the <iPXE script URL> parameter, then the script downloads the iPXE script, extracts the URLs of the boot artifacts generated by assisted installer and the kernel parameters, downloads the artifacts and uses them to install OpenShift.
• hetzner-sno-provision-host-agent.sh: It is also designed to be run in the Hetzner Rescue System. It expects to find the boot artifacts generated by openshift-install agent create pxe-files in /root folder of the rescue environment and uses them to perform the agent-based installation.

How do the scripts work

• Both scripts start by installing kexec-tools into the rescue system, as the key on how both scripts work is to use kexec to replace the kernel run by Hetzner Rescue System (which is currently based on Debian) with the one of the RHCOS PXE live installation media.
• Now it is time to prepare the PXE assets to be used with kexec. Each script does it in a different way.:
  • hetzner-sno-provision-host.sh downloads the indicated iPXE script, which should be the one provided by assisted installer for the discovery phase (as an alternative to the discovery ISO), then it downloads the kernel and the initrd from the URLs on the iPXE script.
  • hetzner-sno-provision-host-agent.sh assumes that the boot artifacts generated by openshift-install agent create pxe-files in ${INSTALLATION_DIRECTORY}/boot-artifacts are copied to /root with their original names (i.e. /root/agent.x86_64-vmlinuz for the kernel, /root/agent.x86_64-initrd.img for the main initrd image and /root/agent.x86_64-rootfs.img for the root file system initrd image). Then, it concatenates /root/agent.x86_64-initrd.img and /root/agent.x86_64-rootfs.img into a unified initrd image on /root/agent.x86_64-combinedinitrd.img because kexec doesn't admit booting with multiple initrd images (unlike GRUB and many other boot environments).
• Last, it kexecs into the downloaded kernel and initrd with the right kernel parameters (hetzner-sno-provision-host.sh uses the parameters provided by the iPXE script, while hetzner-sno-provision-host-agent.sh always uses rw ignition.firstboot ignition.platform.id=metal as recommended by current openshift-install agent create pxe-files output)

Some more warnings

• Both scripts assume that you have previously wiped the hard drives of the node.
• You must be sure to have provided the right network configuration either to assisted installer or the AgentConfig file. As usual with Hetzner, troubleshooting a wrong network configuration may not be trivial and would likely require you to book a KVM console.
• This worked in the server where I could test it. However, as kexec does not perform the hardware initialization in the very same way than a regular boot, it might not work as expected depending on the server hardware (or just on how lucky you are).
• This has not been tested for multi-server installations, but it should work. I'll be happy to learn about bugs or issues specific to trying to install with multiple servers.
• Agent-based installation has been tested so far on connected environments only. If I got some time, I may try to test with proxy or truly disconnected installations (something that should be possible by using firewall outgoing rules), but if somebody does it before me, I'd be happy to learn about possible issues or improvements.