The First Real End-to-End Test Found a Login Error Nobody Had Ever Seen

CaRMen, the small CRM I run for myself, had unit and controller tests from early on, but nothing that drove an actual browser against the actual frontend until October. Setting up Playwright properly — @playwright/test instead of plain playwright, a Chromium-only project, role-based locators, a shared login helper — took one evening. What it found in the first real run had been sitting there the whole time: a wrong password on the login page never showed "Invalid credentials." It just silently reloaded back to an empty login form, every time, with no error visible anywhere.

Why nobody had seen it before

The bug lived in the API client's response interceptor, the code every request runs through on its way back from the server:

if (err.response?.status === 401) {

A 401 meant one thing in this code's mind: your session token is dead, go back to /login. That's correct for the common case — a stored token expiring while you're already using the app. It's wrong for the other case that also produces a 401: the login request itself failing because the password was wrong. There was no token to invalidate yet; the user hadn't logged in. But the interceptor didn't distinguish those two situations, so a failed login attempt triggered the exact same "go back to the login page" redirect that a real session expiry would, overwriting whatever error message the login form was about to show with a fresh, blank one.

As a human using the app, this is nearly invisible. You type a wrong password, the page reloads to login, and the natural assumption is that you mistyped it — which is usually true, so you just try again, more carefully, and it works the second time. The bug only becomes visible when something is watching for the specific text "Invalid credentials" and not finding it, which nothing had been doing before Playwright existed to ask.

The fix narrows the check to the case that's genuinely a session problem:

if (err.response?.status === 401 && err.config?.headers?.Authorization) {

Only redirect when the failed request carried an authorization token in the first place. A login attempt has no token to send, so it never carried one, so a wrong password now just returns its 401 to the caller and the login form gets to show the real error instead of losing it to a reload.

The second bug was in the specs, not the app

Setting up the suite surfaced a second, unrelated problem in how the old ad-hoc test scripts had been written: several selectors used a pattern like locator() || locator(), intending it as a fallback — try the first selector, and if that doesn't match anything, fall back to the second. In Playwright, a Locator object is always truthy, whether or not it matches anything on the page. It's a lazy query, not a result; checking it with || only ever evaluates whether the object exists, which it always does, so the fallback branch never ran regardless of whether the first selector found anything. Several of those same selectors also hit Playwright's strict-mode violations — matching more than one element when the test expected exactly one — which a working fallback pattern would eventually have surfaced anyway, just later and more confusingly.

The rewrite replaced both problems the same way: role-based locators (asking for "the button named Log in," not a CSS selector) that are far less likely to multi-match in the first place, and a single shared login helper that every spec calls instead of each spec carrying its own copy of the login flow, each copy free to have its own slightly different bug.

What a test that drives a real browser is for

Every layer below this — PHP unit tests, Symfony controller tests asserting on JSON response shapes — had been passing the whole time this bug existed. None of them render anything. A controller test can confirm the API returns a 401 with the right error body on a bad password; it has no way to notice that the frontend, on receiving that exact response, throws the error away and shows the user nothing. That gap only closes by running the frontend against the backend and looking at what ends up on the screen, which is the one thing none of the faster, cheaper tests in the suite were doing.

I'd put off Playwright for a while on the usual reasoning that unit and controller coverage was "enough" for a one-person CRM. It wasn't a wrong tradeoff for most of the bugs this app could have — but the specific class of bug it couldn't catch was exactly the one where the backend and frontend each do the right thing in isolation and the wrong thing together, which is precisely the seam unit tests are structurally unable to see across. The hydra-prefix rename a few months earlier was the same shape of bug for the same reason: two halves of one system, each internally consistent, silently disagreeing about what they'd actually agreed to send each other.