I'm blogging about Emacs, Drupal, php and exciting subjects

Cross-encoder reranking: what it costs, and whether you need it

Reranking is usually introduced as the next thing you add after hybrid search. Retrieve broadly, then rescore the top candidates with a model that reads query and document together. Relevance improves, everyone is pleased.

The part that gets skipped is that you have just put a transformer forward pass into your request path — once per candidate, on every search. So let us start with the bill.

Your robots.txt probably doesn't block Google from what you think it does

Here is a robots.txt that appears in some form on a very large number of sites:

User-agent: *
Disallow: /admin/
Disallow: /private/

User-agent: Googlebot
Disallow: /nogoogle/

Read it the way anyone would: everyone is kept out of /admin/ and /private/, and Googlebot is additionally kept out of /nogoogle/.

That is not what it says. Googlebot is free to crawl /admin/ and /private/, and Bingbot is not.

Dependency inversion in Symfony: what autowiring made easy, and what it made optional

Autowiring is one of the better things to happen to PHP. Typehint a class in a constructor, and Symfony finds it, builds it, and hands it over. No configuration, no factory, no container gymnastics. A generation of PHP developers now writes dependency injection without ever thinking about it, which is exactly what good tooling should achieve.

It also quietly made dependency inversion optional, and most codebases have taken the option.

Personas from search logs: what your users actually wanted, in their own words

The standard criticism of user personas is that they are fiction. Someone runs a workshop, the team invents "Marketing Mary, 34, values efficiency, enjoys yoga," and it goes on a wall. Nobody can say whether Mary is right, because nothing about her is falsifiable. She was assembled from assumptions, and she reflects them back with the authority of a laminated poster.

Ansible orchestration: the thing Puppet's model cannot express

Take a requirement that sounds unremarkable: upgrade twelve web servers with no downtime. One at a time, drained from the load balancer first, health-checked before returning to service, and the whole thing stops if more than a quarter of them fail.

Every word of that is about coordination between machines. And that is precisely the thing a per-node convergence model has no vocabulary for.

Measuring search quality: judged sets, nDCG, and knowing whether you improved anything

Here is how relevance tuning usually goes. Someone complains that searching for the company name returns a press release above the About page. You raise the title boost. You try the query. It is fixed. You ship it.

What you do not know — cannot know, from that process — is what happened to the other four thousand queries people run against your search box. Maybe nothing. Maybe you just broke a hundred of them in a way nobody will report, because people who do not find what they want do not file bug reports. They leave.

Hybrid search: combining BM25 with embeddings in OpenSearch

There is a class of search failure you cannot tune your way out of. Someone searches for infrastructure automation; you have an excellent page about configuration management; the words never overlap and BM25 scores it zero. No field boost fixes that, no stemmer, no synonym list you will actually maintain. The term simply is not there.

Drupal Search API with OpenSearch: from setup to relevance tuning

Most Drupal search articles stop at "and now it works." That is the easy half. The hard half is the phone call three weeks later where someone points out that searching for the company's own name returns a 2019 press release above the About page, and nobody can explain why.

This covers both — getting OpenSearch wired up behind Search API, and then actually understanding the scores it produces well enough to change them on purpose.

Provisioning a PHP web server with Ansible: MySQL, PHP-FPM, nginx and Let's Encrypt

Four packages, one playbook. This should be a boring afternoon, and it mostly is — except for one ordering problem that breaks almost every tutorial on the subject, in a way that only shows up on a genuinely fresh server.

So let me start there, because if you understand it the rest is just YAML.

The deadlock nobody warns you about

You write an nginx vhost template with TLS configured, because that is the end state you want:

Subscribe to