A Setup Step That Rewrote /etc/passwd and Took polkit, Wi-Fi and Logins With It

My from-scratch Linux build boots into a KDE Plasma desktop now, and for a while it booted into one with a strange set of problems. Wi-Fi scans in NetworkManager didn't work. Power management didn't work. Login sessions weren't being registered properly. Three subsystems with nothing obvious in common, all broken at once. The cause turned out to be a single build step that rewrites /etc/passwd from scratch.

Reading the journal

The boot journal had the first real clue: polkit.service was failing on every start with "Failed to determine user credentials". polkit is the daemon that decides whether an unprivileged process may perform a privileged action, and a lot of the desktop depends on its answers. On this system, powerdevil, NetworkManager's Wi-Fi scanning and logind's session handling all went down with it, each failing in its own way, and none of their symptoms pointed at polkit.

polkit runs as its own system user, polkitd. The journal was saying that user didn't exist. Further down, systemd-tmpfiles was also complaining that it couldn't resolve the sddm and plasma-setup users. Three system accounts, all missing.

Where the accounts went

They had existed. The build scripts for polkit-1 and sddm-1 both create their accounts when they install, guarded so they only run once:

  • getent group polkitd > /dev/null || groupadd -r polkitd
  • getent passwd polkitd > /dev/null || useradd -r -c "PolicyKit Daemon Owner" -d /etc/polkit-1 -g polkitd -s /bin/false polkitd

The problem is an earlier step. setup-1 follows the Linux From Scratch book's chapter 7: create the base directory layout, the essential files, and the root user. It writes /etc/passwd and /etc/group with a heredoc, cat > /etc/passwd << "EOF", which replaces the whole file.

On a fresh system that's exactly right. But setup-1 isn't only run once. Whenever it ran again after the desktop packages were installed, it put back the book's base set of accounts and silently dropped every account added since. polkit's files were still installed, its service was still enabled, and the user it runs as no longer existed.

That's why the symptoms looked unrelated. Nothing broke in polkit itself or in any of the packages that depend on it. A step that has nothing to do with the desktop deleted an account three layers down.

Why systemd couldn't fix it

On a typical distribution, this exact problem is what systemd-sysusers exists for. Packages ship small declarations in /usr/lib/sysusers.d/, and sysusers creates any declared accounts that are missing, at boot or on demand. The declarations were even present: polkit, sddm and plasma-setup had all installed their .conf files there.

But this systemd is built with -D sysusers=false. That option doesn't just stop sysusers from running at boot. It skips building the tool at all, so there was nothing to call.

The fix: a step that puts them back

Commit e665593 adds a new build step, system-users-1. It runs inside the chroot and recreates the three accounts to match their sysusers declarations, using the same guarded groupadd and useradd pattern as the original package scripts. It's idempotent: accounts that already exist are left alone.

Two details in it are worth keeping. New system IDs are allocated downward from SYS_UID_MAX, so creating polkitd, sddm and plasma-setup in that order gives them the same numeric IDs the original package builds did. But the script doesn't rely on that. It fixes ownership by name: polkit's rules directory goes back to group polkitd, and sddm's home directory is re-owned by sddm, in case it was created while the account had a different ID.

The last line is the one I like most:

find /etc /usr /var /opt /srv -xdev \( -nouser -o -nogroup \) -print

It lists every file still owned by a numeric ID with no account behind it. It doesn't try to fix them. It reports them, because guessing which account an orphaned ID used to belong to is how you end up with the wrong owner on something that matters.

The same mistake, one directory over

The same commit fixes a second bug of the same kind. The kernel build script used to re-derive .config on every build, from the host's kernel config plus whatever modules lsmod showed at that moment. Any change I'd made to the config was lost on the next build, and one re-run had already dropped INET_DIAG without anyone asking it to. The config is now a tracked file, kernel.config, taken from the installed kernel's own config, and menuconfig saves changes back to it.

Both bugs have the same shape: a step that rebuilds some state from a template every time it runs, in a system where other steps have since added to that state. The step is correct in isolation and destructive in sequence. What made the /etc/passwd version hard to find was the distance between cause and symptom: it deletes a file entry, and what you see is a Wi-Fi applet that won't scan.

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
Please share this article on your favorite website or platform.