The version of a database upgrade that fits on a disk with limited free space and the version of a database upgrade that keeps a true instant rollback available turned out to be two different operations. Choosing the one that actually fit meant giving up the other, deliberately, with the tradeoff written down plainly rather than discovered the hard way if it were ever needed.
What a normal, safe upgrade looks like
The straightforward way to upgrade a database to a new major version is to build the new version's data files as an entirely separate, independent copy, leaving the old version's files completely untouched throughout. If anything goes wrong, or the new version needs to be abandoned for any reason, the old copy is sitting there exactly as it was — a real, complete rollback path requiring nothing more than switching back to it. The real cost of this approach is disk space: for the duration of the upgrade, and for however long afterward the old copy gets kept as insurance, the disk needs enough free room to hold two full copies of the database at once.
Why that wasn't available here
This database's disk didn't have that much room to spare — nowhere near enough free space to hold a second complete copy of a database over two hundred gigabytes in size alongside the original. The upgrade tool's alternative mode avoids that requirement by hard-linking data files between the old and new clusters instead of copying them — the new version's data directory reuses the same underlying files the old version already has on disk, rather than duplicating them, which is what made the upgrade possible on this box's actual available space at all.
What sharing files instead of copying them actually costs
A hard-linked file is shared, not duplicated — both the old and new cluster genuinely point at the same physical data on disk, right up until either one writes to it. The moment the new, upgraded cluster starts running and modifying its own data — which is the entire point of actually using it — those shared files begin to diverge from what the old, unmodified cluster still expects to see. From that point forward, the old cluster's data directory is no longer a faithful, unmodified snapshot of anything. It's a data directory that used to be current and has since had files silently changed underneath it by a different process using the same underlying storage.
The honest consequence, stated plainly rather than glossed over
This wasn't presented as a minor caveat buried in a longer list of upgrade notes. It was named directly, as a real property of the chosen approach: the old data directory is not a safe instant-rollback path once the new cluster has started and taken real writes. Reaching for it after that point wouldn't produce a clean, working restoration of the pre-upgrade state — it would produce a data directory in an unknown, partially-overwritten condition, which is arguably worse than having no rollback path at all, since it could look usable without actually being trustworthy.
What the real fallback actually was
The genuine rollback plan, accepted from the start rather than assumed to exist by default, was a full database dump taken immediately before the upgrade began — verified twice, independently, before the upgrade was allowed to proceed. If the new version ever needed to be abandoned after real writes had already happened, the real recovery path was restoring that dump into a freshly built, clean instance from scratch, not flipping back to a data directory that could no longer be trusted to represent any single consistent point in time.
Why naming the tradeoff mattered more than avoiding it
The choice to fit this upgrade into the disk space actually available, rather than freeing up enough room first for a fully safe approach, was a real, deliberate tradeoff — not an oversight discovered only if something had gone wrong later. Writing down exactly what kind of rollback safety was and wasn't available, before the upgrade started rather than after, is what turns a real limitation into an accepted risk instead of a false sense of security. A rollback plan that quietly doesn't do what "rollback plan" usually implies is far more dangerous than having no rollback plan clearly labeled as such — the failure mode isn't the missing safety net, it's believing one exists right up until the moment it's needed and doesn't work.
Leave a comment
Comments are reviewed before they appear.