There is a class of search failure you cannot tune your way out of. Someone searches for infrastructure automation; you have an excellent page about configuration management; the words never overlap and BM25 scores it zero. No field boost fixes that, no stemmer, no synonym list you will actually maintain. The term simply is not there.
Embeddings solve exactly that failure, and introduce a mirror-image one of their own. Which is why the answer in production is almost never "replace keyword search with vector search" — it is running both and merging the results.
The two blind spots are complementary
BM25 matches tokens. Its weakness is vocabulary mismatch: paraphrase, synonymy, and the long natural-language questions people increasingly type, where the words the user chose and the words the author chose have no intersection.
Dense vector retrieval matches meaning. You run every document through an embedding model, get back a few hundred or few thousand floating-point numbers positioning it in a semantic space, do the same to the query, and return the nearest neighbours. Paraphrase stops mattering because you are no longer comparing words.
Its weakness is the exact opposite one. Ask a dense retriever for ERR_CONN_RESET, or a part number, or a Danish surname, and it will cheerfully return semantically adjacent documents that do not contain your token at all. Embedding models are trained on meaning, and an error string does not have much. This is why "just switch to vector search" produces a system that feels smarter on discursive queries and noticeably stupider on the precise ones — which are often the queries that matter most, because someone typing an error code knows exactly what they want.
Neither method is a superset of the other. That is the entire case for hybrid retrieval, and it is why it has become the default across OpenSearch, Elasticsearch, Weaviate, Vespa, Qdrant and Milvus alike.
Why you cannot simply add the scores
The obvious implementation — run both, add the numbers, sort — does not work, and the reason is worth understanding because it determines which fusion method you should pick.
BM25 scores are unbounded positive values. As covered in the BM25 article, they depend on collection statistics, query length and document length; a score of 8.7 means nothing in isolation and is not comparable to a score of 8.7 from a different query. Cosine similarity, meanwhile, is bounded — typically landing in a narrow band, where the difference between a great match and a mediocre one might be 0.89 versus 0.81.
Add those together and the BM25 leg dominates entirely, not because it is more relevant but because its numbers are bigger. Scale them to a common range and you have a different problem: normalisation is sensitive to outliers, so one anomalously high score drags the whole distribution and quietly changes the balance between your two retrievers, query by query.
OpenSearch gives you two ways to fuse
Both are implemented as search pipeline processors that intercept results from a hybrid query — a compound query holding up to five subqueries, each scored independently at shard level.
The normalization processor arrived in 2.10. It rescales each subquery's scores using min-max or L2 normalisation, then combines them with an arithmetic, geometric or harmonic mean. This is score-based fusion, and it can be very good once calibrated against your data.
The score ranker processor arrived in 2.19 and implements reciprocal rank fusion. It ignores scores entirely and looks only at positions.
Start with RRF. It requires no calibration, it cannot be distorted by an outlier score, and it is stable when one retriever's score distribution shifts underneath you — which happens every time you swap embedding models, reindex, or change an analyser chain. Score normalisation can beat it, but only once you have a judged evaluation set proving it does on your corpus.
RRF, briefly
For each document, sum the reciprocal of its rank in every result list it appears in:
score(d) = Σ 1 / (k + rank_i(d))
i
where rank_i(d) is the document's position in the i-th list, and k is a smoothing constant that damps how much the top few positions dominate.
The behaviour that makes it work: a document ranked third by BM25 and fifth by the vector search beats one ranked first by BM25 and absent from the vector results. Consistency across retrievers is rewarded over dominance in one. That is usually what you want, because a document both methods like is a document that is both lexically and semantically on-topic.
The constant defaults to 60 in both OpenSearch and Elasticsearch, following the original 2009 paper by Cormack, Clarke and Büttcher — which noted the choice was not critical, with performance barely moving across roughly 10 to 100. Do not spend an afternoon tuning it. Leave it at 60.
Wiring it up
Four pieces, in this order: an ingest pipeline that turns text into vectors, an index with somewhere to put them, a search pipeline that fuses, and a query that asks for both.
The examples below assume you have a text embedding model registered and deployed in ML Commons, and that you have its model ID. Setting that up is its own topic — the OpenSearch docs cover both an automated workflow and a manual path — but everything after it is the same regardless of where the model runs.
1. The ingest pipeline
This is the piece that generates embeddings automatically as documents are indexed, so nothing in your application has to know about vectors at all:
PUT /_ingest/pipeline/embed-pipeline
{
"description": "Generate embeddings for article body text",
"processors": [
{
"text_embedding": {
"model_id": "YOUR_MODEL_ID",
"field_map": {
"body": "body_embedding"
}
}
}
]
}
field_map maps source field to vector field: read body, write the embedding to body_embedding. Test it before you index anything real:
POST /_ingest/pipeline/embed-pipeline/_simulate
{
"docs": [
{ "_source": { "body": "hello world" } }
]
}
You should get back the document with a body_embedding array of floats attached. If you get an error instead, it is almost always the model ID or a model that is registered but not deployed.
One trap that will bite a Drupal site specifically. Fields named in field_map are mandatory — if a document lacks one, indexing that document fails outright. Drupal content types are full of optional fields, and a node with an empty body will take down its own index request. Add "ignore_failure": true to the processor, or make sure the field always exists even when empty.
2. The index
PUT /articles
{
"settings": {
"index.knn": true,
"default_pipeline": "embed-pipeline"
},
"mappings": {
"properties": {
"title": { "type": "text" },
"body": { "type": "text" },
"body_embedding": {
"type": "knn_vector",
"dimension": 1024,
"method": {
"name": "hnsw",
"engine": "faiss",
"space_type": "cosinesimil"
}
}
}
}
}
default_pipeline is what makes this automatic — every document indexed into articles now runs through the embedding pipeline without the client asking for it.
Two things that cannot be changed later without a full reindex: dimension must match your model's output exactly, and the vector field name must match the field_map target. Decide on the model before creating the index, not after.
3. The search pipeline
PUT /_search/pipeline/rrf-pipeline
{
"description": "Hybrid retrieval with reciprocal rank fusion",
"phase_results_processors": [
{
"score-ranker-processor": {
"combination": {
"technique": "rrf",
"rank_constant": 60
}
}
}
]
}
Per-subquery weights are available through a parameters.weights array if you later find evidence one leg is systematically better. Do not set them on day one.
4. The query
GET /articles/_search?search_pipeline=rrf-pipeline
{
"query": {
"hybrid": {
"queries": [
{
"multi_match": {
"query": "infrastruktur automatisering",
"fields": ["title^3", "body"]
}
},
{
"neural": {
"body_embedding": {
"query_text": "infrastruktur automatisering",
"model_id": "YOUR_MODEL_ID",
"k": 50
}
}
}
]
}
}
}
Note that the second clause is a neural query, not a raw knn one. You hand it the query text and OpenSearch embeds it for you using the same model that embedded the documents — which is the point. If you embed the query with a different model, or a different version of the same model, the vectors live in incompatible spaces and your results become quietly meaningless. A raw knn clause taking a literal vector array is the right choice only when you are generating embeddings client-side and taking responsibility for that consistency yourself.
Repeating the model ID on every request gets old. A neural_query_enricher request processor lets you set a default model per index or per field, after which the model_id line can be dropped from queries entirely.
Two retrievals run in parallel, the pipeline fuses them, you get one list. The k on the vector leg is how many neighbours to retrieve before fusion — larger gives the fusion more to work with at some latency cost, and 50 to 100 is a sensible starting range.
Getting embeddings without a US API
Every hybrid search tutorial assumes an OpenAI endpoint. For a Danish or EU public-sector site that is often the point at which the project stops, because embedding a corpus means sending every document you own to a third-country processor, and the resulting vectors are derived data with an uncomfortable relationship to the original text.
You have three options that avoid this entirely, in increasing order of effort.
Run the model inside OpenSearch. ML Commons can host a model in the cluster itself, so text never leaves the search infrastructure. Operationally this is the simplest thing that can possibly work, and it means one fewer service. The cost is that inference competes with search for the same JVM heap and CPU, which is fine for indexing throughput measured in hundreds of documents and not fine at scale.
Run it as a separate service you host. An embedding endpoint on your own infrastructure, called through an ML Commons connector. More moving parts, but you can scale and restart it independently, and it is the arrangement that survives contact with a real content pipeline.
Use a European inference provider. OVHcloud and Scaleway both offer model endpoints under EU jurisdiction, which keeps the GDPR analysis short without you operating GPUs. Worth confirming the specific processing terms rather than assuming EU incorporation settles it.
Which model
The multilingual open-weight families — E5 and BGE among them — are the usual starting point for Danish content, and produce 1024-dimensional vectors in their larger variants. English-only models are substantially better on English and substantially worse on everything else, which is the trade to be deliberate about rather than accidental.
Check current standings on the Scandinavian embedding benchmarks rather than trusting a model card or, for that matter, this paragraph — the leaderboard moves faster than anyone's blog post, and a model that was clearly best a year ago rarely still is.
What it actually costs
Memory, and this is the number that surprises people. HNSW graphs live in memory, not on disk. OpenSearch documents the estimate as:
1.1 × (bytes_per_dimension × dimension + 8 × M) bytes per vector
M is the maximum number of bidirectional links each node keeps in the graph, defaulting to 16 — so 8 × M is the edge overhead at 8 bytes per link, and the 1.1 is roughly ten percent slack on top.
The only variable you control cheaply is the first term, which is simply how many bytes each dimension occupies:
| Vector type | Formula | Per vector at 1024 dims |
|---|---|---|
| float32 (default) | 1.1 × (4 × d + 8 × M) | ~4.6 KB |
fp16 (sq encoder) | 1.1 × (2 × d + 8 × M) | ~2.4 KB |
| byte | 1.1 × (d + 8 × M) | ~1.3 KB |
Work an example through at the default. A hundred thousand documents at 1024 dimensions is about 460 MB of resident memory doing nothing but holding the graph — on top of your existing index, on top of the JVM heap you already sized. Ten thousand documents is a comfortable 46 MB. A million is 4.6 GB, and now you are having a conversation about node sizing.
Then multiply by your replicas. Every replica is another complete copy of the graph, resident on another node. One replica doubles the requirement. This is the step most people skip when sizing from a document count, and it is the difference between a cluster that fits and one that does not.
Quantisation is the lever, and it is a better deal than it sounds. Adding the sq encoder to the field's method parameters switches you to 16-bit vectors and halves the memory more or less exactly. The obvious worry is recall, and OpenSearch's own benchmarking on a 100-million-vector dataset at 768 dimensions measured a 47.6% memory reduction with identical recall at k=1000 — four data nodes instead of eight, no measurable quality cost. Indexing throughput was generally better as well.
Byte vectors quarter it again, but only if your values genuinely fit the signed 8-bit range, which is a property of your embedding model rather than a setting. Product quantisation goes further still with a real and measurable recall cost; that is a decision to make against an evaluation set, not a default to reach for.
The 3.x line has invested heavily here — 3.4 added native FP16 vector scoring and memory-optimised search warmup — so check the current documentation before concluding the raw float32 numbers are what you have to live with.
Latency. Two retrievals in parallel cost roughly the slower of the two plus fusion overhead, not the sum. The genuine addition is embedding the query at search time, which means a model inference call on every single search. If that model is a network hop away, that hop is now in your search latency budget.
Chunking, which nobody warns you about. Embedding models have a token limit, commonly 512. A 3,000-word Drupal node does not fit. Feed it in anyway and most implementations silently truncate, so you have embedded the first two paragraphs and indexed that as the meaning of the whole article. You need a chunking strategy — split on headings or paragraphs, embed each chunk, index chunks as separate documents or as nested vectors, and decide how chunk hits roll up to a single result. This is the part of a hybrid search project that actually takes the time.
You have to measure it
Hybrid search demos beautifully and does not always help. On corpora with consistent controlled vocabulary — technical documentation, product catalogues where everyone uses the manufacturer's terms — BM25 alone is sometimes already at the ceiling, and you have added considerable machinery for no gain.
Build a judged set before you build the pipeline: 50 to 100 real queries pulled from your logs, each with the documents you consider relevant marked by hand. Then measure nDCG@10 and Recall@100 for BM25 alone, vectors alone, and the fusion.
Break the results out by query type — short keyword queries, natural-language questions, exact identifiers. The aggregate number hides the thing you need to see, which is whether hybrid helped the discursive queries without damaging the precise ones. If your identifier queries got worse, weight the lexical leg up. That is the moment weights are justified, and not before.
On the Drupal side
The contrib ecosystem has moved faster here than you might expect: there is a semantic extension to Search API OpenSearch built on the semantic field type added in OpenSearch 3.1, an OpenSearch NLP module wiring up ML Commons for semantic and hybrid retrieval, and an OpenSearch vector database provider for Drupal's AI module.
They are young, and at least one is explicitly experimental with no stable release. If this is going near production, my honest advice is to prototype against OpenSearch directly with the queries above, confirm hybrid actually beats your current BM25 on your judged set, and only then decide how much of it to push back through Search API. The measurement is the hard part and it does not require the Drupal integration to exist.
What does carry over regardless is everything you already did to the keyword side. Field boosts, analysers, language handling — none of it becomes obsolete. BM25 remains one of the two legs, and a hybrid system with a badly tuned lexical retriever is a hybrid system running at half strength.