LFS takes the wrong approach in my opinion. I did it once manually over a couple of days. I suppose I hoped enduring the pain of waiting for the packages to compile would somehow make me smarter. It didn’t. I didn’t learn much. It was like following a baking recipe. I know I need to add eggs and sugar in the ratio it says but I’d be damned if I could tell you why that specific ratio works.
It’s just too much to take in at once. You start by building a cross compilation toolchain but… why? And you add the bootloader dead last, even though it’s the first thing you see when you boot a Linux machine?
I reckon a better approach would be to build it step by step, only adding new things once you’ve discovered a need for them. First you get the bootloader working, then you get it to actually load the initramfs and kernel, etc… you could even create your own extremely simple initramfs first before adding dracut, etc…
This gets you to a system you can boot and play with asap. And then you can slowly start to discover why you actually need all these things through exploration and experimentation.
LFS helped me a lot. It taught me how to compile packages manually, and how GNU/Linux components fit together.
Armed with that knowledge, I could do things others could not do:
- Make applications that bundle/automatically download their dependencies, compile them with custom flags (static linking)
- Diagnose build errors buried in deep dependencies when using vcpkg
- Compile programs for esotheric systems: e.g: Intel MacOS X 10.5, an obsolete system noone else on earth is using except the company I worked at
- Create application targetting obsolete systems but using the latest version of its dependencies
- Compile recent programs on esotheric systems, by bootstraping a newer gcc on the newer system, patching as necessary
- Create my own stripped down distro, tuned to my specifications
- Use that stripped distro as a docker container
- Run recent linux on obsolete hardware no longer supported by mainstream distros
- Reformat a cloud dedicated server without physical access by first bootstraping a minimal distro on a ramdisk, unmounting the main drive, partitioning that drive, and installing the new distro on that drive. Reboot and pray.
- Build a cross-compiler for targetting a freestanding environment
- Build latest KDE/XFCE for debian stable. No more "but debian ships with outdated software!!11"
- Build applications on a distro they were not designed for (Zimbra OSE on debian)
Learning how to compile packages manually it was a common skill long before Linux came to be, and how GNU/Linux fits together could also easily be learnt with Slackware or Yggdrasil.
As someone who started using Linux in 1996 (Slackware, of course), I learnt the hard way about compiling packages. I remember trying to compile the kernel, and doing "make config" and following hundreds of options, and not knowing that I then needed to do "make" or "make zimage" or "make install" after any of the config stuff. Obvious, dumb, rookie errors, but still understandable for a teenager coming from Windows. Doing LFS really taught me a lot about dependencies and complexity, as well as understanding of the actual boot process. All of this helped me greatly in later years.
Your experience sounds exactly like mine. I downloaded Linux 2.6 which took 2 days to compile. Then when I installed it on my Slackware system completely hosed it. But learned so much…
>Learning how to compile packages manually it was a common skill long before Linux came to be
Right. I used to compile many open source packages from source on many flavors of Unix back in the day, for years. Perl, Python, MySQL, many others. Miscellaneous utilities. Esoteric languages. Even Oracle, once, on a Unix server, right at a large manufacturing company site. Even building Unix kernels from source was common on some Unix versions. And not just for a regular install. Many times, (re)building Unix kernels from source, was the only way to get some software packages or hardware devices or peripherals, like printers or terminal controllers or even hard disk controllers, to work on the system. Typically you would edit some config files or settings files manually, change values of params, or in some cases use a TUI to do that, and then rebuild via a makefile. You had to save the old kernel in case the new one didn't work, and if so, lather, rinse, repeat, till everything did work.
And that practice was nothing special. A lot if people did it routinely. Most, in fact, I'd say. Talking about devs and sysadmins here, not end users, although some of the latter probably did it too.
Many apps, tools and utilities came as source code, often in a .tar or .tar.gz file. You would untar or gunzip it into a new directory, and do your stuff, running makefiles, other shell commands, setting env. vars., etc.
None of that requires LFS; I compiled KDE via scripts many years ago. Why would LFS be needed for that? That makes no sense. Debian being slow is their problem, other distributions aren't snail-like like that. The default compiler used in LFS is also hardly a real cross-compiler, it is more a bare-minimum compiler for C/C++. This annoys me because e. g. I need fortran for R, so a bare-minimum compiler isn't as useful as a more feature-complete GCC. And GCC also causes some issues, e. g. like my problem with options.cc.
I think you missed the main argument though - LFS isn't quite as good as it could be. That is a correct statement that was made.
LFS is not OS development. You build a distribution by assembling all the components. If you want to develop an OS from scratch, perhaps you're looking for osdev.org.
> It was like following a baking recipe. I know I need to add eggs and sugar in the ratio it says but I’d be damned if I could tell you why that specific ratio works.
I know a french cookbook[0] which contains not only recipes for dishes, but also for some of the intermediary ingredients, like stocks, that can now be bought at the supermarket.
I feel that I’ve learned a lot by following those recipes, be it simply in the structure of french cuisine.
And to me, LFS is a cookbook in a similar sense.
I've learned the most from LFS long time ago, when I had no internet. I had a mismatching set of Slackware and redhat source CDs, and LFS itself didn't match the versions on CDs too. Since then I can build anything and anywhere, even if it's not cooperating :)
> And you add the bootloader dead last, even though it’s the first thing you see when you boot a Linux machine?
That actually makes perfect sense. If you did the bootloader first, how are you going to build it? How do you know what to boot from which filesystem on what device? By the time you've got your root filesystem complete, it's easy to tell.
If you do things step by step incrementally, you keep having to go further back in the process. Starting over repeatedly gets tedious, boring and frustrating. And it'll take ages to finish the complete OS.
Agreed, it's near the end of the Gentoo installation as well. If you forget to install GRUB, just chroot back into it and continue.
If I installed the bootloader first, I'd have to remember to run dracut after the system was installed. Or I install it last, create the init at the same time and make a habit of it.
Except that if you fail to install the bootloader - which happened to me many times - then you do not have a working system. So really the prior argument makes sense.
IMO the guy is right - LFS could be way better. But it has this current structure and does not deviate from it. That's the bad part.
> If you do things step by step incrementally, you keep having to go further back in the process. Starting over repeatedly gets tedious, boring and frustrating. And it'll take ages to finish the complete OS.
Pray tell how that is necessary for the bootloader? The chain used in LFS isn't the only one that makes sense or is required. IMO it would indeed be better to have a system you boot into as quickly as possible, on real hardware, then continue from there.
If you're going to run the OS, you have to make sure it compiles and runs the bootloader and its installer. The toolchain could be different enough for that to fail. And you do boot into a reasonably small OS and continue from there. That's what BLFS is all about.
As pointed out already, LFS is a cookbook, and you learn most by having an idea about how you want to tune your Linux system.
So at points you want to dive deeper, you do — and go off the beaten path.
It requires a desire not just to understand how bits and pieces fit together, but how you can make them work even better for yourself and what is and is not customizable?
Arguably, even if building it today is easier — back when I did it in 2000, you sometimes struggled to find where the canonical source code of the package is, not to mention compilation errors, linking errors, etc. Only then did you get to configuring the init RC system, choosing between them (no systemd yet), etc.
But a requirement is — IMO — that you are strongly opinionated about how a system should serve you, and look for opportunities to tune it.
This should be said more often, and in many other contexts as well.
Most educational resources are what- and how-based, only the best ones are primarily why-based, what Wirth once wrote in the preface to some Pascal book (I paraphrase) 'discussions are motivated, so that the learner gets coverage and understanding, instead of memorization'.
> I did it once manually over a couple of days. I suppose I hoped enduring the pain of waiting for the packages to compile would somehow make me smarter. It didn’t. I didn’t learn much.
Exactly my experience. Back in the day when all I knew was Windows and a couple of BSDs I wanted to get into Linux, because it seemed like it was going to become very relevant in the next decades. After years of slacking I thought LFS would probably be the ideal choice. Turned out it wasn't, at least for me. I ended up "learning Linux" with distros like Gentoo, Arch and later even Ubuntu, and by just replacing my production systems with Linux, which were still running on OpenBSD mostly. I have revisited LFS a couple of years ago and honestly didn't take much from it. I get the appeal, but it's not for everyone.
Exactly my experience. I tried LFS several times, but I didn't learn much. I learned more with Arch Linux, but I truly appreciate the work that was put in LFS, marvelous for the right type of learner.
Sounds like a speedrun of the tutorial. How did you approach packaging then and what system did you use to compile everything?
I did the tutorial in the late 2000s I think and it took me 4-6 months if I remember correctly. At the time I wasn't completely new to Linux but setting up things like the boot loader was quite tricky for me.
For me the main motivation was getting a deeper understanding, being able to fine tune and customize my system. Of course I didn't go blindly through the tutorial but only did steps that I understood. (In case something goes wrong) Also I started to use (x)stow for everything.
It is possible to do it over a few days really, even with not too much knowledge.
Requiring four months seems as if it was a side project, which is of course totally fine, but for any main task it really should not take more than a few days, even on a semi-slow computer (well, depends on how slow).
Well as mentioned it was in the 2000s. Compiling KDE alone took several days. Also there was no automation/script, so you really had to go to ftp.gnu.org and look in which folder the coreutils package is ;) I realize there's now a list of packages with checksums
I did it in a VM took me 20 hours. Most of that time was waiting for things to compile and fixing mistakes I made because I didn't read the instructions properly.
I learned a lot doing Gentoo stage 1. I was also following the guide like a recipe, but at the time I hadn't been exposed to the underlying machinery that is often automated, abstracted away, or done for me already. It was also a good test on how well I could follow instructions and later on apply my own customisations. Like lego.
Except you can write bootloader only if you have something to boot.
Bootloaders as we know them came after OSes, because the need of a bootloader is to boot something.
Following your logic of building a piece only if needed, a bootloader should come after.
I like the recipe analogy and I actually think adding things only if needed will help cutting unuseful features and possibly to have different implementations for solutions you need.
No point in reinvent the whole thing, tho.
Following your analogy, if all cakes need egg, there must be a reason even if we don't know it.
You probably want to use a working bootloader to see if your ‘hello world’ OS-fragment works at all. No one is writing much OS without running it, surely.
in the bad olden times you definitely started to write your operating system for your new computer from the first instruction executed upwards. it wasn't at all common to have a comprehensive virtualization/simulation environment.
until very recently you couldn't even do OS development without a boot loader, so asserting that that would be the last thing you would write is just nonsensical
oh. I'm talking about authoring OS and boot loaders from an absolute blank slate. like starting from the world where you toggled them in from switches, and later by burning proms
I second this a thousand million times. But with AI, i can't help but admit i find myself indulging in building things backwards, now that i can try a couple different approaches simultaneously for it.
I used CLFS (Cross-Linux From Scratch) once to bootstrap Linux on my SGI Indy, and I learned quite a lot from this. That was back in 2008 or so, so way before AI and extensive online guides for almost everything. It taught me how cross compilers work, what dependencies exist in the GNU toolchain, how the SGI boots, and much more.
Too bad Cross-LFS is no longer a thing. Then again, with all these microcontrollers and ARM and RISC-V systems today, cross-compiling is no longer as magic (and tricky) as it was back then...
well, if you happen to be in a 'rm -Rf /' situation (which with agentic AI seems to be more often the case) to know how to operate in a chroot environment could be helpful
The step by step is the right approach... but you'll discover that many 'components' are _INSANE_ and that includes their SDK.
Bare LFS is not enough (I run my own): you need a kind of userland (above glibc) multi-version system which some kind of "atomic-ish" switching (always have a stable SSH running in case something goes wrong, better than unplugging the system disk and fix it on another computer).
For linux, same thing, with a kind of flip-flop-ing for updates/fix/etc.
The insight that gives you will probably scare you: the current "open source" stack is an abomination (the worst is the SDKs I think).
That's why we need _LEAN_ open source, and that includes the SDKs.
If you think that's an abomination, try corporate billed-by-the-hour enterprise software where nothing ever changes unless it's approved by several layers of management.
I promised myself not to work for any of those anymore :)
Well, in this very case, it is more complicated than that: they have to protect their software against planned obsolescence and developer tantrum (it would be the same for any "in production software").
Modifications and changes must be weighted very carefully. For instance "removing" (including in the SDK) is usually a less worse modifications than
the others and it decreases the global technical size and complexity/number of layers of the stack/SDK (which is top priority in "security"), namely most of the time but not everytime a good thing.
Because, on most complex software (open source or not), a "believed" benign modification can be disastrous. It is even worse if close to bare metal involving complex hardware (even with massive QA[testing]).
Free software doesn't typically have an SDK. A compiler is just one of the available programs. So is a text editor, a shell and script interpreters, some incarnation of make, etc. There's nothing really special about the compiler and you can write perfectly fine programs without ever touching it.
That said, port trees usually have devel-* sections. Does that count?
To some extent I agree with you. I'd like to decouple some parts,
e. g. the tutorial part is great but it should ideally be distinct
from "just go at it" 1:1. I also fail to compile various things right
now, which is frustrating; two or three years ago all went well. The
whole LFS build up is too brittle. In part this is due to external
complexity, but LFS needs to re-think what it does. It's also sad that
they are now systemd-only, which further adds intrinsic uselessness. I
don't need anything systemd offers; all my boot up logic is stored in
yaml files and autogenerated to target formats via ruby as-is.
Compiling the packages doesn’t teach much it’s all the glue in between. No? It’s the startup scripts and the configs and creating a kernel and all the things.
It’s on my list of things to do when I have some free time
You know, you could also just read about each step to learn more. I mean if you feel you're following a baking recipe, you can just... read about it more. And then have LFS as a simple way to experiment if you want to change anything or not. Also, technically you don't need to have a bootloader installed on your main hard drive if you already have it on another disk.
I mean, eh... last time I checked, this was Hacker news. Yesterday, there was an Ask HN topic on how to enable dark mode on HN. Is this the new hacker culture? Complaining that the resources merely point in some direction instead of specifying details?
It’s just too much to take in at once. You start by building a cross compilation toolchain but… why? And you add the bootloader dead last, even though it’s the first thing you see when you boot a Linux machine?
I reckon a better approach would be to build it step by step, only adding new things once you’ve discovered a need for them. First you get the bootloader working, then you get it to actually load the initramfs and kernel, etc… you could even create your own extremely simple initramfs first before adding dracut, etc…
This gets you to a system you can boot and play with asap. And then you can slowly start to discover why you actually need all these things through exploration and experimentation.