My Alpine Linux Bootloader project is a success! I can now boot any Linux rootfs such as Debian over VPN + NFS via Alpine. The entire bootloader creation uses 200 lines of shell scripts.
The advantage is that the target OS needs not to be pre-configured for PXE, VPN or NFS. In a regular netboot setup, each target OS's initramfs must know how to do networking / mount rootfs. WireGuard is used here, so each needs customization. Using the AlpineBL approach, the bootloader takes care of everything. I used a boot disk in my setup, not PXE, but they're not exclusive, PXE can deliver that boot disk if you want.
I'll publish a full tutorial on Alpine Wiki, but here's a quick review.
First, create an Alpine rootfs at ./mnt with alpine-make-rootfs, with only 4 packages preinstalled: alpine-base, linux-lts, wireguard-tools, kexec-tools. You need another DHCP client for IPv4, I use IPv6 SLAAC so I skipped it. Enable some OpenRC services such as "devfs", "modules", and "local" in that chroot. Extract the kernel from rootfs. Create an initramfs with cpio at ./mnt. Base image done.
Next, create another rootfs overlay at ./rootfs_extra, with /etc/wireguard/wg0.conf and /etc/local.d/99-alpinebl.start. Chain two initramfs binaries with cat. Linux unpacks them sequentially (feature+bug: ./rootfs_extra can overwrite existing files). Overlay image done. Warning: this image contains private keys.
Boot your machine with that Alpine kernel, and your custom initramfs, with rdinit=/sbin/init (force the kernel to use the real userspace init as the "init script"), retain_initrd (don't release compressed initramfs), kexec_load_disabled=0 (bypass Alpine hardening), alpinebl.stage=1 (custom flag).
Alpine userspace starts in a normal runlevel. OpenRC's "local" service executes our custom boot script 99-alpinebl.start. In the custom boot script, load network card kmod, enable networking, enable WireGuard, mount NFSv4 to /mnt. In theory we can switch root now, but we're still using the Alpine kernel, not the target kernel which can be arbitrary. We need to "swap" to the target kernel, and overlay more target kmods in initramfs on the fly.
To do this, create another initramfs using files in /mnt/lib/modules and /mnt/lib/firmware via cpio + gzip. Chain the original initramfs (accessing via /sys/firmware/initrd with retain_initrd) and the new overlay to /dev/shm/initrd Run kexec with the /mnt/vmlinuz symlink and /dev/shm/initrd. Run kexec with alpinebl.stage=2, and strip retain_initrd since we don't need it in Stage 2.
A small warning here: at the end of Stage 1, there are 4 copies of initramfs in RAM: /, /sys/firmware/initrd, /dev/shm/initrd, and kexec buffer, each is 1 GiB+, if you don't have RAM, you need to delete blobs under /lib/firmware before cpio / kexec but after modprobe. If RAM is still not enough, you need to stop using retain_initrd, instead, instead, find your own boot drive's block device, mount it, and find the initramfs - tedious work).
The system starts again, and all the procedures described above is repeated again: enable network, enable WireGuard, mount NFSv4. Now the startup script sees alpinebl.stage=2 in /proc/cmdline, so it knows to switch_root instead of kexec.
But remember, we're in an initramfs, but the boot script is not executed by /bin/sh with PID 1. This is a full system with Busybox init + OpenRC, we're a normal userspace process, so we can't kill or replace init unless we're init already... We're stuck here. Should we use a PID namespace to work around it? No!
Busybox init supports hot update via the "restart" target after receiving a SIGQUIT (an arcane feature), it's a replacement binary path. We create a /init file (to bypass switch_root's safety check), write ::restart:/sbin/switch_root /mnt /sbin/init to /etc/inittab, send SIGHUP to PID 1 to reload config, then send SIGQUIT to PID 1. So init shutdowns userspace daemons, call execvp(), and switch_root takes over.
Alpine userspace is gone. Debian userspace starts up with systemd like a regular boot. Existing Ethernet, WireGuard interfaces, and the NFS mount are auto-inherited by systemd. You get a login shell. Mission complete. #alpine #alpinelinux #NFS #WireGuard
