My search engine has a small side feature: a gallery of people it has come across, grouped by what they're known for. Artists, musicians, politicians, journalists, and so on. On 2026-09-12 I noticed that the Artist bucket, a few entries away from Picasso, contained a well-known 20th-century dictator.
I'm not going to name the person here, and the commit that fixed it doesn't either. The person isn't the interesting part. What's interesting is that the data said the right thing, and my code read it in the wrong order.
Where the category comes from
The category comes from Wikidata. Every person entity there has an "occupation" property, P106, and a lot of people have several claims under it. Someone can be a lawyer, then a journalist, then a politician, and Wikidata records all three.
My PersonOccupationCategorizer mapped occupation QIDs to gallery categories. "Painter" (Q1028181) mapped to Artist. "Politician" (Q82955) mapped to Politician. The categorizer walked the person's P106 claims in array order and returned the first one that matched a known category.
For this person, Wikidata lists both a "painter" claim and a "politician" claim, and the politician claim is what they're known for. The painter claim happened to come first in the array, so the loop returned Artist and stopped.
The field I'd been ignoring
Wikidata already solves this. Every statement has a rank: preferred, normal or deprecated. Preferred means "this is the one that matters when there are several". Deprecated means "this is recorded, but known to be wrong or superseded". The politician claim was ranked preferred. The painter claim was normal.
My parser's type annotation didn't even include the rank key. I'd modeled the claim as just a value and treated array order as if it carried meaning. It doesn't. Array order is insertion order, which reflects when someone edited the entity, not how significant each claim is.
The fix
Commit 75c6ca2 changes the loop to make two passes: preferred claims first, then normal ones. Deprecated claims are skipped entirely. Within each pass, the first claim that maps to a known category still wins, so the behavior for people with one occupation, or several of equal rank, is unchanged.
foreach (['preferred', 'normal'] as $wantRank) {foreach ($claims as $claim) {$rank = $claim['rank'] ?? 'normal';if ($rank !== $wantRank) { continue; }// ... map the occupation QID as before}}
A missing rank defaults to normal, which matches how Wikidata treats statements without an explicit rank.
Two new tests pin it down. One builds exactly the shape that caused the bug, a normal-rank painter claim followed by a preferred-rank politician claim, and asserts Politician. The other puts a deprecated "actor" claim ahead of a normal "musician" claim and asserts Musician.
Fixing the rows already in the database
Fixing the code didn't fix the gallery. Everyone already categorized kept the category they'd been given. The existing app:person-candidates-recategorize command only filled in rows where the category was still null, because it had been written as a backfill.
The same commit adds --force to it, so it can re-run the mapping over rows that already have a category. That's the difference between "new entries will be right" and "the gallery is right". Without it, the dictator would have stayed next to Picasso until someone noticed and fixed that one row by hand, and every other misfiled person would have stayed misfiled.
Why this kind of bug survives
For most people the first matching occupation is the right one, so the bug was invisible in aggregate. Spot-checking a dozen entries would almost certainly have shown a gallery that looked fine. It took one entry where the wrong answer was jarring enough to notice.
That's the uncomfortable part of consuming structured data from a source like Wikidata. The data model is richer than the obvious reading of it. Rank, qualifiers, start and end dates: they all exist to resolve exactly the ambiguity a naive "take the first value" parser runs into. Ignoring them doesn't make the parser simpler. It means the code is making those calls by accident, based on edit order.
Leave a comment
Comments are reviewed before they appear.