The Linux From Scratch book is a list of commands to type, in order, over a few hundred pages. Typing them once is the point of the exercise. Typing them again after a mistake in chapter 8 is not, so my build drives the whole thing with make: one make at the top runs everything from downloading sources to the last package, and after any failure, running make again picks up where it stopped. Three small ideas make that work.
1. A stamp file per package, named after its version
make rebuilds a target when it's out of date, and the GNU make manual defines that precisely: a target is out of date if it doesn't exist or if it's older than any of its prerequisites, by modification time. A package build doesn't produce one obvious output file to use as the target, so each package gets an empty stamp file that its rule touches when the build script succeeds.
The trick is the stamp's name. For the first GCC pass, the rule looks like this:
STAMP = .built-$(gcc_v)$(STAMP): gcc.sh ../binutils-1/.built-$(binutils_v)GCC_V=$(gcc_v) ... ./gcc.shtouch $@
Three things now trigger a rebuild, all through make's ordinary rule. Editing gcc.sh makes the script newer than the stamp. Rebuilding binutils gives it a newer stamp, which is a prerequisite here. And changing the pinned GCC version changes the stamp's file name, so the target simply doesn't exist yet. Nothing in the scripts has to check versions itself.
The versions live in one file, versions.mk, included by every package's Makefile and by the downloader. That's what made it possible to pin coreutils back to 9.10 for a patch with a one-line change: the new version name meant a new stamp, and the next make rebuilt coreutils and the packages whose stamps are chained after it, and nothing before it. In the toolchain stage each package's stamp depends on the previous one's, so the chain is the build order.
2. Setup steps are stamped too
Not everything is a package. Chapter 4's directory skeleton and the chapter 7 change of ownership are one-off setup steps, and at first the skeleton was a manual step outside make. After a full reset, a plain make stalled at the start of the toolchain because the tools directory didn't exist. Now the skeleton has its own stamp, lfs/.built-skeleton, and runs once per build tree.
The same goes for the recursive chown that hands the tree to root. It walks the whole tree, so it's stamped and runs once per reset, not on every make.
The chroot mounts can't be stamped, because they don't survive a reboot. Instead, each mount is guarded with mountpoint -q, which exits 0 only if the directory is already a mount point, so mountpoint -q lfs/proc || sudo mount ... is safe to run any number of times.
There's one ordering rule in there with a comment worth quoting in spirit. The step that fixes ownership of the tree's root and its mount-point directories runs before anything is mounted on them, and skips any that are already mount points: changing the owner of a directory with the host's /dev bind-mounted on it would change the host's /dev.
3. Resets that keep the downloads
The cleaning targets are layered by how much they throw away:
make clean: unmount the chroot's filesystems, delete the built system and all stamps, keep the sourcesmake clean-sourcesandmake clean-downloads: also drop the tarball cachesmake distclean: all of the above
Downloads go to a cache first, are checksummed, and are then copied into the build tree's sources directory by a populate-sources step, only for the files the current scripts actually use. So after distclean, a plain make rebuilds everything unattended, and after a plain clean, it rebuilds without downloading anything.
Why bother
A from-scratch build fails often, and usually late: the coreutils patch alone took four rounds over eleven days. With stamps, each retry costs one package, not a rerun of everything before it. With layered resets, starting over is a decision about how much to keep, not a manual cleanup. None of this is clever. It's make doing the one thing it's for, deciding what's out of date, with file names chosen so that "out of date" means what I want it to mean.
The full build, in order, is in the Linux From Scratch overview.
Leave a comment
Comments are reviewed before they appear.