Deploying the removal of a Drupal module looks like it has an obvious order: take the module out. In practice it's two separate operations that happen to both look like "removing a module" from the outside, and doing them in the wrong order left production in a broken half-state I had to clean up by hand.
Two different layers, one word
Locally, removing drupal/advagg went: drush pm:uninstall first, while the module's code still existed on disk, then composer remove to delete it. That order matters because pm:uninstall is a database migration in the literal sense: it runs the module's uninstall hooks, which delete its schema, its configuration objects, and any data it owns, and it does that by calling into the module's actual PHP code. The code has to be present for that step to work at all. Only once that's done does the code stop mattering, and only then is it safe to delete it.
drush config:import is a completely different kind of operation, even though it also changes what's stored in the database. It doesn't run any module code and it doesn't care whether the module is currently installed or not; it just makes the core.extension table and every other config object match whatever the YAML files in sync/ say they should be. If the sync files say a module isn't installed, config import will happily write that, whether or not anything has really cleaned up after it first.
Those are the two operations I was conflating: a database migration that depends on code being present to execute correctly, and a config import that doesn't care whether the code exists at all. Treating them as interchangeable steps in a "remove the module" checklist, reorderable based on which one happens to run more conveniently in a given deploy script, is exactly the mistake that bit me.
What happened on production
On production, the deploy ran composer install, which deletes a removed package's files, before the database there had really had the module uninstalled. Composer doesn't know or care that a module needs pm:uninstall run against it first; it just reconciles the filesystem with the lock file. The result: production's core.extension config still listed advagg as installed, its config objects were all still present, but the code those config objects and hooks depended on was simply gone. Any code path that tried to reference the module, an include_once for its files, a service definition pointing at a class that no longer existed, was now a live landmine instead of a settled removal.
This is the same "phantom module" state I'd already hit twice earlier in the same session with google_tag and showcase_lite: config claiming a module is installed while the code backing it has already been removed from the codebase. Recovering from it meant going in after the fact and doing manually, out of order, what should have happened automatically in order: deleting the orphaned config objects directly via drush php:eval/php:script, then stripping the module's key out of core.extension so Drupal's own bookkeeping matched reality again.
The actual fix
The fix isn't a smarter deploy script that retries in the right order after the fact. It's sequencing the two kinds of operation correctly to begin with, on every environment, not just locally:
- Sync
sync/and run the uninstall, viadrush pm:uninstallordrush config:importagainst config that already reflects the module being removed, while the module's code is still present - Only after that step has fully completed, run
composer installso the code disappears
Locally this order was easy to get right by habit, because I was doing each step by hand and could see the module still needed to exist for the uninstall to work. On production, where the deploy is a script running composer install as one of its ordinary steps, nothing enforced that same sequencing. The script had no idea that this particular composer change depended on a database-level step happening first, because from composer's point of view it's just another dependency change like any other.
Why this keeps being worth writing down
The underlying trap isn't specific to advagg, or even to module removal generally. It's the assumption that "changes persistent state" is a single category of operation you can freely reorder. A database migration that runs code and a config import that just writes declared state are both, technically, "changing what's in the database." Only one of them stops working once the code it depends on is gone. Knowing which is which, and sequencing a deploy around that distinction rather than around what's most convenient to script, is the real lesson, not "remember to run things in the right order," which is advice that's true and useless in roughly equal measure until you know exactly which order that is and why.