Hybrid search runs a keyword search and a vector search over the same data, then merges the two result lists into one ranking. It is a common way to improve retrieval for search boxes and for retrieval-augmented generation (RAG), where an LLM answers from documents your system fetches first.
The idea is simple: each method is good at something the other is bad at. Combining them gives better results than either alone.
Why one search method is not enough
Keyword search (also called full-text search) matches the words in a query against the words in documents. It is precise. It finds product codes, error numbers, names, and rare terms exactly. But it has no idea that “laptop will not turn on” and “device fails to boot” mean the same thing.
Vector search compares meaning. A model turns the query and each document into embeddings, and the system returns the documents whose vectors are closest. It handles synonyms and paraphrases well. But it can blur exact details. A query for ERR_4021 may bring back pages about ERR_4012 because the two look alike to the model.
The two retrievers inside hybrid search
A hybrid system needs two indexes over the same documents.
The keyword side usually scores documents with BM25 (Best Matching 25). BM25 rewards a document when:
- it contains the query terms, and more often is better, with diminishing returns
- the terms are rare across all documents, so they carry more information
- the document is not much longer than average, so long pages do not win just by size
The vector side embeds the query with the same model used for the documents, then ranks by a distance measure such as cosine distance. See vector representations for how text becomes vectors and vector distance metrics for how closeness is measured. Large collections use an approximate index, such as DiskANN or HNSW, so the search stays fast.
Merging the lists with Reciprocal Rank Fusion
Now there are two ranked lists. The obvious move is to add the scores, but that does not work. BM25 scores have no fixed upper limit, while vector similarity or distance lives on a different scale. One side would swamp the other.
Reciprocal Rank Fusion (RRF) avoids the problem by ignoring raw scores and using only rank positions. Each document earns a small amount from every list it appears in:
RRF score = sum over lists of 1 / (k + rank)
Here rank is the document’s position in a list (1 is best) and k is a smoothing constant, commonly 60. A document ranked high in both lists gets the highest total. A document in only one list still gets a partial score.
Take four documents, A to D, with k = 60:
- The keyword list ranks A first, C second, B third. D is absent.
- The vector list ranks B first, A second, D third. C is absent.
A and B appear in both lists, so they beat C and D, which appear in only one. A narrowly wins because it is first in one list and second in the other.
Weights
You can make one retriever count more. A weight multiplies a list’s contribution. If the vector list is weighted 2 and the keyword list 1, each vector term becomes 2 / (k + rank). In the example, B (first in the vector list) then scores about 0.04866 and edges past A at about 0.04865. D, which only the vector list found, overtakes C. Final order: B, A, D, C.
Use higher vector weight when queries are conversational. Use higher keyword weight when users search for exact names, codes, or identifiers.
Hybrid search in practice: Azure Cosmos DB
Many engines support hybrid search, including Azure AI Search, Elasticsearch, OpenSearch, Weaviate, and PostgreSQL with pgvector plus full-text search. Azure Cosmos DB for NoSQL is a clear example because a document can hold its text and its vector side by side.
Setup has four steps:
- Enable the vector indexing and search feature on the account.
- Create a container with a vector policy, a full-text policy, a vector index, and a full-text index.
- Insert documents that have both a text property and a vector property.
- Run hybrid queries.
A vector policy describes the embedding field. The dimensions must match your embedding model, and the distance function should match the metric that model was trained for:
{
"vectorEmbeddings": [
{
"path": "/vector",
"dataType": "float32",
"distanceFunction": "cosine",
"dimensions": 1536
}
]
}
The indexing policy then adds a full-text index on the text path and a DiskANN vector index on the vector path. Excluding the vector path from the regular index keeps writes cheaper.
A hybrid query calls RRF inside ORDER BY RANK, combining a VectorDistance score and a FullTextScore score:
SELECT TOP 10 *
FROM c
ORDER BY RANK RRF(
VectorDistance(c.vector, @queryVector),
FullTextScore(c.text, "reset", "password"),
[2, 1]
)
The last argument is the optional weights array. Here the vector score counts twice as much as the BM25 score. Your application must embed the user’s question first and pass the result as @queryVector.
One limitation to plan for: vector policies and vector indexes cannot be changed after the container is created. Choose the dimensions and distance function carefully, or you will need a new container. Check the current Azure Cosmos DB hybrid search documentation for the latest syntax and limits.
Where hybrid search fits in RAG
In a RAG system, retrieval quality limits answer quality. If the right document is not in the top results, the model cannot use it. Hybrid search raises the chance that the right passages arrive. Those passages then fill the model’s context window, so how many you return also matters for context engineering.
Some teams add a reranker after fusion. A reranker is a slower model that reads the query and each candidate together and reorders them. Hybrid search fetches a broad, good candidate set cheaply, and the reranker polishes the top of it.
Common pitfalls
- Adding raw scores. BM25 and vector scores are not on the same scale. Fuse by rank, as RRF does, or normalize carefully.
- Mismatched embeddings. The query and documents must use the same embedding model, and the distance metric should match what that model expects.
- Ignoring text analysis. Language, stemming, and stop-word settings change keyword results. Set the language of the full-text index to match your content.
- Guessing weights. Start with equal weights, then tune on a set of real queries with known good answers.
- Assuming hybrid always wins. It usually helps, but it adds an index, storage, and latency. Measure it against keyword-only and vector-only search on your own data.