I replaced Grub w/ rEFInd a few years ago and I've never looked back. The ability to just scan filesystems and boot images is so simple/easy compared to the Grub menu edit and "reinstall" process you have to go through every time you want to make a change. I've never used systemd-boot but highly recommend rEFInd over Grub.
I also have a Python script for installing Ubuntu on native ZFS w/ encryption and using rEFInd. You can use this script from a live boot environment. If nothing else, can easily review for how to do it manually.
I concur. Refind is pretty much universal for me: it detects Linux kernels automatically, windows, boots from USB or detect USB (I always have an emergency copy that will boot my systems if I screw up), it also runs on Intel macs...
Also... It is very customizable with good a 100% graphic UI.
systemd-boot is also pretty hands-off. My distro's (OpenSUSE Tumbleweed) kernel packages build regular initrd images in package postinstall, but since the postinst script uses dracut I just had to drop a dracut config (`uefi=yes`) in /etc to make it build UKIs in /efi instead. systemd-boot automatically picks up those UKIs.
I just have to remember to delete the UKIs manually when I uninstall the corresponding old kernel package, otherwise the EFI partition would fill up eventually. If my distro ever switches to UKIs by default then the postuninstall script would handle that automatically.
I also switched to REFind because I wanted to boot my various Linux installations from encrypted ZFS pools, but after realizing how much more convenient it was (because it can just boot from whatever bootable OS installation it finds by scanning the disk), now I always use it instead of grub, regardless.
I noted recently that it also sees systemd-boot installations and will boot from those, if desired (in my case it was a fresh installation of Pop OS, which uses systemd-boot by default).
For me, it's unambiguously better (meaning easier, with no extra periodic steps required to maintain my ability to boot whatever bootable thing I want), but I am curious about what scenarios would make one want to use systemd-boot (or grub) instead.
> I am curious about what scenarios would make one want to use systemd-boot (or grub) instead.
I have Linux VMs which boots in ~2s (to ssh login; ~700ms to userspace). I use systemd-boot because it has nicer integration with my distro of choice than bare EFI stub, and it adds very minimal overhead in boot time.
Use linux-image-cloud; use a modern hypervisor with no legacy devices (only virtio etc); use tiny-initramfs; mask systemd-networkd-wait-online.service; add cryptomgr.notests noreplace-smp to kernel command line.
I think that's all, and use a moderately fast CPU (i7-8700K which is already 5 years old).
I'm talking about the time taken by OMVF/SeaBIOS or whatever equivalent in the VM; which is why I asked about the type of VM/Hypervisor because that might not apply.
I put a bunch of live environment/rescue images (trinity, UBCD, hirens, systemrescue etc) on a usb flash with ventoy, and ventoy couldn’t boot any of them except for systemrescue (seemed like it had an issue with EFI with the other ones)..
As an alternative to ventoy, can I use refind on a usb flash drive with iso images?
I also used to use rEFInd while I was running macOS/windows dualboot for a short time. I am going back to it from grub now since booting from my zfs root partition with grub has become a hassle.
Years ago on an arch system I would boot directly from UEFI to the kernel (using a feature known as EFISTUB[1]) Ever since then I have not understood why most distros don't switch to this setup. It would seem like removing an entire bootloader from the boot sequence would significantly improve boot times, which was my experience with that setup at the time.
Does anyone know why anyone would choose systemd-boot over something like efistub?
IIRC, systemd-boot relies on EFISTUB. It's just a boot manager, not a boot loader. One EFI application calling another.
For trivial setups (one kernel that keeps getting updated in-place), a boot manager is not required. For more sophisticated setups (version in kernel file name or Nix style system versions or whatever), a boot manager is very convenient.
Yeah, exactly. This is why you still need to sign kernel images if using Secure Boot with systemd-boot.
Personally the only reason I have it in the loop is it makes it so much easier to switch the an lts kernel if say... zfs has failed to build on a kernel update and I didn't notice (not that THAT would ever happen). Could use multiple EFI entries for that, but systemd-boot is simpler.
Fair enough this type of thing was my main worry with that setup, and I kept a backup known-good configuration around manually for this purpose. Any automation there seems useful.
I'm using an "Unified Kernel Image" directly through the EFI boot-manager.
It's the most simple boot setup I've ever used, and it doesn't have the mentioned issues:
> the same disk is now impossible/harder to boot in another system/mainboard.
Just go and select your UKI in the EFI boot-manger. Works on every modern system OOTB.
> no more choice between different kernels in case the newest one broke something
You can have more than one UKI around. You can, for example, have even one with a whole recovery system included in the initrd part!
> to change the cmdline with kenel parameters you now need to run efibootmgr
You only need to recreate the UKI with an updated command line. No change to the boot-manager needed.
Also a UKI has a better security story compared to all other options. You have only one signed EFI binary. No need for complex setups that validate the initrd after the fact, as everything is included in the UKI and signed as a whole.
Have this setup on my Debian box but used actually the superior documentation form the Arch wiki to get there.
> I'm using an "Unified Kernel Image" directly through the EFI boot-manager.
I just recently switched all of my Arch boxes to this setup. The first was so I could enable Secure Boot on that machine, the rest just because it felt so much simpler and cleaner than Grub. Actually, the last one just stopped booting entirely with Grub but worked just fine with UKI.
> You can have more than one UKI around. You can, for example, have even one with a whole recovery system included in the initrd part!
Same! I was considering writing up how to do this for the Arch wiki since it’s handy and I couldn’t find any complete guides.
As long as you don’t delete the old kernel, you can recover by creating a known good entry every few updates. The arch article also has an ego shell you can install that got me out of a few unbootable cases with a failed Nvidia dkms.
You could always bake the cmdline into the kernel itself, but that would mean recompiling the kernel each time. (Although if that's the only change, that shouldn't take too long)
I also am a huge fan of EFISTUB. EFISTUB is basically the only way of making secure boot work in a sane way for Linux - include the initramfs in the kernel and sign the whole thing with your secure boot keys. You don’t even need to compile the kernel, you can use dracut to package everything up yourself.
Except if the boot manager implementation of the UEFI is bad. For example on Intel NUCs you cannot use one-shot functionality and kernel command line at the same time. If you put systemd-boot in between it works.
My old mobo would refuse to accept NVRAM changes, overwrite them at boot or something, then hardcoded try to launch the windows bootloader from the ESP. I had to trick it by naming my bootloader the same thing as the Windows bootloader. Signing new things for secure boot didn't work at all but luckily it could be disabled.
This was an MSI motherboard in case anyone's wondering.
If it's for recovery, etc. then that may not be a real issue?
EDIT: Anyway, I'd probably only ever rely on an external USB stick, but I don't know how that interacts with Secure Boot, tbh. I use Ventoy + loads of images of popular distros. Can fit quite a few distros on a 128 GiB USB stick! :D
As long as the bootloader/kernel image is signed, I don't think it matters to secure boot whether or not it was loaded off a USB stick. Presumably the firmware implementing the secure boot check must copy the image into RAM before verifying it anyway, at which point the USB storage doesn't matter.
Of course, the firmware may be configured to refuse to boot a USB drive as a matter of policy.
Well, one reason you'd need one is if your boot filesystem is encrypted and you want to be able to reboot the machine remotely. (I use ZFSBootMenu for this purpose; it's accessible via SSH and can be used to decrypt the filesystem and choose what to boot.)
There are hardware solutions to that problem, but most regular PCs don't come with a remotely-accessible boot menu.
That's not my experience. I've only ever seen HP do an actual file browser in UEFI to pick an arbitrary .EFI file, but overall every computer and laptop I've seen has a button you can press to pick from a list of operating systems available to boot.
You might not think of it as a boot menu, but every uefi system I have ever used has a menu you can bring up by holding a key during boot that lets you choose which disk to boot from. In reality this menu has all the entries you set up in efibootmgr. You can also usually see this menu from the save and exit page of the setup/bios page. These entries are about as customizable as grub entries via efibootmgr.
I didn't say boot loader, I said boot _menu_. The boot loader may be a mandatory part of UEFI (otherwise the entire thing wouldn't actually work...), but there is absolutely no mandate for having any type of GUI, not even a boot menu.
Most devices out there will always boot the first entry (or BootNext) no questions asked. Some devices may allow reordering boot loader entries in their setup program. Almost no devices will actually a) pause booting whenever there are multiple valid entries b) show you a menu where you can choose one of alternatives c) respect your timeout value, choosing the first boot entry if the timeout expires. Incidentally these 3 happen to be the basic properties of what people usually demand from a boot manager.
And some devices are simply hardcoded to _always_ boot \EFI\BOOT\BOOTX64.EFI, no matter what. EFI implementations are all over the place.
It's gotten somewhat better, but it's hard to rely on anything.
But we got decorative flames in the firmware setup dialogues now.
Flames make computer go fast.
A Lenovo or HPe system won't have that, of course. But I've had plenty of "consumer" boards for my gaming rigs, or handled friends gaming mainboards.
It's not all flames of course. But the design of the firmware UI certainly got really flamboyant when UEFI hit the scene while quality of implementation really suffered.
It's gotten somewhat better, but now it's getting hard to buy non-enterprise hardware without over 9000 RGB LEDs on there. Sigh.
At least the LEDs don't bother my bootloader.
Both URLs I sourced directly mention a menu. Here's the first:
> UEFI allows the consolidation of boot menus from the OS loader and platform firmware into a single platform firmware menu. These platform firmware menus will allow the selection of any UEFI OS loader from any partition on any boot medium that is supported by UEFI boot services. An UEFI OS loader can support multiple options that can appear on the user interface. It is also possible to include legacy boot options, such as booting from the A: or C: drive in the platform firmware boot menus.
The whole point is that I don’t want it to pause boot to show me a menu unless the system is broken. Boot should not need to be interactive by default. But every uefi laptop and desktop device I have ever used has a key you can press during boot to bring up a boot menu.
Yeah, this was my one concern with the setup. But to be fair, it's not a flaw with efistub, it's a flaw with the kernel update logic in arch. I manually added an entry for a known good configuration which I periodically updated manually, but I know of no reason why the kernel update logic couldn't do that for me like it does in grub. And as a fallback you can install a UEFI shell option to boot into which works in emergencies.
As a distro I wouldnt want to rely on uefi implementations that range from ok to garbage tier (no cmdline arguments, hidden boot selection menu etc)
I want to enter a sane booting environment as soon as possible (gummiboot, refind, grub). Also why not? Its basically free and In my experience doesnt really complicate secure boot.
> Ever since then I have not understood why most distros don't switch to this setup.
The persistence of GRUB for x86-64 systems is both fascinating and irritating; the original driver for users paying the price of the complexity of GRUB was the idea that it would bring consistency across architectures and be an overall reduction in complexity for distro maintainers. Instead the bootloaders for non-x86 architectures are generally not-GRUB (zipl, various ARM options which are sometimes SoC specific, and so on), while x86 users are stuck with the byzantine complexity of GRUB.
I have my taylor-made kernel compiled with all and only the drivers I need, without an initram and with EFISTUB, directly called by the UEFI as its first boot option. That's how I manage to boot in 5 seconds.
Then I have some distro-supplied kernels, with initram, modules, non-standard parameters AND grub, just for those times when I need something exotic. I have to stop the normal boot with Esc and select the second boot option from the UEFI menu, that points to grub. It's clumsy and slow but I just need it once in a blue moon.
systemd-boot presents a boot menu at startup. It can support different boot targets defined in files under the EFI directory, and it automatically detects Windows UEFI targets.
Even if you almost never reboot your box it can go into deep sleep when it's not used. It doesn't "heat the planet" significantly when doing so.
Actually it would be interesting to calculate whether frequent reboots don't use more energy in sum than putting the machine into sleep. Does anybody have numbers on that?
In mobile phones similar optimizations where done at least when battery sizes were well below 1000mAh. I think race to sleep was very competitive. So you run at full power for a short time and then you go to a deep sleep mode.
Discaimer: Have not worked with mobile phones for 10+ years.
Typical boot is somewhere.. okay, lets take 30 seconds for all boot shenanigans. Okay, 1 min for ease of count.
That's 0.0694444444444444% a day.
Doubt it.
Even monster servers (like 4CPU and TBs of RAM) boot up in 10 minutes and that's only 0.69% of the day.
Modern systems don't gobble all up all available power at the startup and pretty quickly go to a power controlled mode.
Sure if you have some monster with a dozens of HDDs then there could be some nuances, but at that point you surely don't reboot often anyway
Fun fact about systemd-boot, you have to hold down the spacebar to get the boot prompt, and it remembers the last boot. So if you let your toddler bang on the keyboard while booting, and you're boot looping into the BIOS config screen, hold down space at the next boot and maybe everything will work out OK.
Pop!OS uses it on anything with a UEFI secure boot setup which was something of a surprise to me since I didn't notice anything except that I don't see the GRUB menu at boot time, and to dual boot I have to drop to the bios or use the Windows 11 restart bollocks to get back to Pop!OS
Although there is a reddit comment that I think was quite apropos: LILO got too complex so we got GRUB, now GRUB is a beast and we have systemd-boot. Not sure what other advantages it might have over GRUB other than perhaps it plays nicer with Windows/UEFI. I still prefer GRUB on my other machines where I'm still allowed to use it.
It's a genuinely complex problem. Booting x86-64 with arbitrary hardware attached while maintaining compatibility with boot modes from 30+ years ago is quite a feat.
On some devices, I see the line "EFI stub: load initrd from command line option" (or similar) as just another line on the same text screen that prints early UEFI messages.
And I'm still wondering about that and how text-mode/console output works.
I get it when I'm attached via a serial port, but... how many console rendering implementations for VGA output are on my system?
The biggest blocker for me using systemd-boot is that there's no signed EFI binary yet so you can't use Secure Boot (unless you want to enrol your own signing key, which is inconvenient and I'm not even sure is possible on cloud providers.) If you use Ubuntu, Canonical provide a GRUB binary signed by Microsoft's shim.
Here is a question for you, what value is secure boot providing you that you cannot get from an encrypted root filesystem with a TPM and systemd-cryptsetup.
That is too say, the signed grub binary can be used to load literally anything. It can even chain load into another non-signed modified grub.
On the other hand, the crypt setup route ensures that you bios, grub, TPM, and many other measurements are correct.
Secure boot doesn't prevent code from doing whatever it wants once it's running. It prevents the system from starting to run code that isn't trusted in the first place. That's still necessary even if you have a TPM. If you're storing your disk encryption key in a TPM, there needs to be an API for the OS init routine to access the key from the TPM. You can't depend on any secrets outside of the TPM before decrypting the storage (otherwise you wouldn't need a TPM), so instead you need to guarantee that the code accessing the TPM's API isn't malicious. Typically, once the OS reads out the storage encryption key, the OS then immediately asks the TPM to disable its API until a subsequent boot. From that point on the kernel has the key in RAM, and there's no way for unprivileged code to access it (other than bugs). Secure boot provides the mechanism to make that possible.
If you don't have secure boot enabled, someone could boot a malicious image that has access to the TPM's API before it's disabled, and therefore gain access to your storage. They would still need to get that image on your computer of course. As you say, though, the vast majority of software running once the OS is booted isn't necessarily secure.
Secure boot alone doesn't really help much for foiling bad actors with physical access, since you can just go into the bios settings and disable it. That's why bios typically have their own password protection. On some "embedded" computers, it's practically impossible to turn secure boot once it's been enabled; there's literal fuses inside the chip that get blown.
The TPM api isn't disabled after boot, and things still use it the computer starts up. IE, it's possible to use the TPM for ssh/gpg keys. Other things like esims and password managers can still use them.
It also isn't something that any software can use to just spit out the secrets it stores. The whole point of the TPM is that it will only release information if the measurements of the system match. IE, if I chainload grub->window that will cause a different measurement then it the system just booted windows.
But again, secure boot will only run signed images, but a signed grub can run anything. Also grub and it's configuration must be on an unencrypted partition. You can easily tamper with its config to load whatever you want.
>but a signed grub can run anything. Also grub and it's configuration must be on an unencrypted partition. You can easily tamper with its config to load whatever you want.
No, it implements the verifier API which needs to be disabled for this to be true. This should not be the case for distributions utilizing the shim+grub setup (Ubuntu/SUSE/Fedora/Debian/etc)
I mean the API for accessing the disk encryption key is disabled, not that the whole TPM is disabled. I think we're both saying the same thing, just with different terminology. The TPM will only give the CPU the key when it asks if the current string of measurement matches what the TPM expects. The OS changes the measurement before it starts running untrusted code, and so the TPM no longer gives the same access.
I think the critical point is that the measurements are made by the CPU, not the TPM. Malicious code running on the CPU can tell the TPM whatever it wants to tell it. The TPM is notionally just a peripheral hanging off a low speed serial bus. All the TPU can do is watch the sequence of measurements being fed to it, and decide whether or not it looks suspicious. The idea is that everything is chained off of an initial known to be trustworthy state. The initial known state is provided by boot-up, where the first code to run comes from ROM or an EEPROM or flash chip. Secure boot is the mechanism by which that boot-up code can load more code off a storage device and authenticate it. Secure boot is literally what keeps the system secure while booting, before the TPM is configured.
Your point about grub being insecure in some ways is likely true. It's been around a long time, and existed before secure boot. The grub image itself being on an unencrypted partition doesn't matter, though, because it is signed; secure boot won't run it if it isn't authentic. I don't think the configuration matters much either. Grub can be configured to give the TPM a measurement of what it's actually loading, i.e. the kernel and initrd and boot options. The TPM can then refuse to give up its secrets if that measurement doesn't match what it expects.
Booting is complicated, and when you're mucking around with stuff like grub, it's easy to accidently break the chain without realizing it. There's a reason why corporate IT tends to prefer Mac, Chrome OS, or Windows based computers; it's easier to say that the system is properly secured through boot until the corporate spyware is running. As a practical matter, though, a broken link or two doesn't mean that the whole chain is useless. Stuff like secure boot, TPM, etc. all decrease the attack surface. You shouldn't store nuclear detonation codes on a computer you know isn't perfectly secured, but it's probably fine for your tax return from 2015 and meme collection; no one's going to bother individually targeting you.
I wish systemd-boot had implemented a BIOS shim somehow, since we're still stuck with grub for most cloud providers and other non-UEFI capable setups. Basically something that BIOS boots into a UEFI loader-esque so it uses the same path for everything, similar to how ARM can "UEFI boot".
FWIW on AWS, all nitro instance types can boot as UEFI if the AMI has itself set to use UEFI and all arm64 instances only support UEFI. So if you stick to modern instance types, you can happily use UEFI in AWS.
GCP looks like it does UEFI too. And Azure Gen2 instances use UEFI too.
It was a failed experiment overall, too much recompiling even for me, however one thing I did get out of it was that I discovered I didn't need a bootloader.
You can boot Linux as an EFI executable.
Granted, you cannot pass any dynamic boot arguments to it, and initramfs is out of the window it seems (or I just never got it working), but it was remarkably easy to add the commandline options I wanted into the kernel at compile time and just call it as an executable automatically from the EFI BIOS of my motherboard.
Is there any way to add a 'Shutdown' and 'Reboot' menu entry to systemd-boot? Like you can with Grub? This has been holding me back from switching all my machines to systemd-boot.
Mainly because I have an Optiplex micro hidden behind my monitor, where I use the keyboard to power on. Sometimes accidental power-ons happen so it's good to have that menu entry.
I think encrypting your boot partition and kernel is not very useful. The "better" (IMO), would be to sign the kernel and use secureboot, but otherwise leave it unencrypted. This way the kernel and whatnot can't be tampered with and you don't have to deal with an encrypted boot partition.
I don't think there would be anything in your kernel or bootloader that needs to be confidential!
Well, kernel version is already some information. Whatever it gives. It's not about guessing how useful that information is. It's about minimizing it by default.
In a systemd-boot setup there is no "boot partition", encrypted or otherwise. There is an EFI partition that contains the systemd-boot EFI binary and the UKI binary, and it is a regular unencrypted FAT32 partition.
The UKI will contain kernel + initramfs to be able to mount your encrypted root partition as `/`, using regular cryptsetup, not the special low-perf code used by grub.
And to be clear, this is how UEFI boot would work even if you didn't have systemd-boot and had your firmware directly boot the UKI.
Hm, I don't want to put the kernel into EFI partition though. With GRUB the kernel sits in the encrypted /boot on the root. The catch is that GRUB can decrypt root on its own without the kernel, but it's super slow for some weird reason. So I need something similar, but without performance bottlenecks.
It's advantageous for the process of trying a passphrase to be too slow to do a million times a second but fast enough that it isn't bothersome to the user. This is a tunable parameter --iter-time given in ms when adding or modifying a passphrase to a slot.
If I understand correctly its cryptsetup luksChangeKey DEVICE --iter-time <time in ms>
Consult your manual and backup before making any changes to encrypted devices lest tragedy ensue.
2 things of note.
- Luks slots are tried in succession adding a second faster slot after a slow slot will be slower because it will try the slow one first fail then unlock with the fast slot.
- You should back up your luks header because a small amount of corruption in the right place can make the entire encrypted volume unrecoverable
Not that slow. When kernel decrypts LUKS it's fast because it's using modern CPU commands that GRUB doesn't for some reason, if I understand correctly. Which just results in way worse decryption time.
Wrong question. Why not? I want everything to be encrypted. Something must be left out and it should be the least possible amount. Leaving out the kernel is clearly not the least possible.
That's the only one that's there unless you have custom firmware for the motherboard. So with UEFI, GRUB being left out is pretty small in comparison with leaving out the whole kernel.
But it's not so easy. I actually looked into all the dependencies of a Linux system and it turns out to be an extremely entangled monolith at the core. Everything depends on everything. So you actually can't have "just a kernel and systemd as 'user-land'".
> actually looked into all the dependencies of a Linux system and it turns out to be an extremely entangled monolith at the core
Any resources for this ? Most books I've read/seen are either about Linux kernel internals or using userspace Linux. I'd like to know more about what actually goes into making a complete usable distro.
Ever tried setting 'init=/bin/sh' on the command line? That gets you a shell (or any other program you care to run) and from there a little creativity is all it takes.
I pretty much did this too but in Arch Linux. It's just a script[1] that I run and I have it working with the linux and the linux-lts kernels. It's pretty nifty. It just works on Arch Linux since kernels don't have version numbers on the name I never have to touch again after installation.
If you're lucky and your machine supports coreboot there is even better way to boot into linux - use slimmed down linux kernel as a direct payload of coreboot and kexec into your actual kernel. This way you're skipping any disadvantages of grub (if you can name any which matter) and don't need to deal with relatively new and not as much battle tested code to boot your trusted kernel.
No, just an intermediate kernel is written into flash memory along with coreboot. It doesn't wear little flash chip as it doesn't write anything which needs to persist, that's actually one of the points of this solution. Its sole role is to chain-load your destination kernel. So you just basically move your bootloader into non trivially writable memory (as it can be write-locked) to make it more tamper proof.
There is always a risk of bricking your system if you're not careful, but wearing off flash isn't one of those risks if that's the concern.
Nope; bootloader flash chips use NOR instead of NAND flash precisely to prevent this.
NOR flash guaranteed for a minimum of 100,000 complete rewrites of every cell, with no flash translation layer gimmickery. The downside is that it is very expensive, which is why a few dozen megabytes of NOR flash costs as much as many gigabytes of MLC NAND.
Typical NOR flash chip; "Minimum 100,000 Program/Erase Cycles":
NAND flash vendors often advertise "up to" a similar number of P/E cycles on their SLC devices, but you'll have a hard time getting them to guarantee that. And for TLC and MLC NAND you find in non-industrial products the number is more like 3% of that:
I also have a Python script for installing Ubuntu on native ZFS w/ encryption and using rEFInd. You can use this script from a live boot environment. If nothing else, can easily review for how to do it manually.
- https://www.rodsbooks.com/refind/
- https://wiki.archlinux.org/title/REFInd
- https://github.com/rsyring/zfs-on-root/blob/master/zor.py