HQAI Engineering HQ
Search
Checking sign-in...

Hybrid search

Concept9 min readintermediate

What you’ll understand

  • Explain why keyword search and vector search each miss results the other finds
  • Calculate a Reciprocal Rank Fusion (RRF) score for a small set of ranked results
  • Describe how a hybrid query, its indexes, and its weights are set up in a real database

Helpful first: Vector representations for AI systems · Vector distance and similarity metrics

Hybrid search runs a keyword search and a vector search over the same data, then merges the two result lists into one ranking. It is a common way to improve retrieval for search boxes and for retrieval-augmented generation (RAG), where an LLM answers from documents your system fetches first.

The idea is simple: each method is good at something the other is bad at. Combining them gives better results than either alone.

Why one search method is not enough

Keyword search (also called full-text search) matches the words in a query against the words in documents. It is precise. It finds product codes, error numbers, names, and rare terms exactly. But it has no idea that “laptop will not turn on” and “device fails to boot” mean the same thing.

Vector search compares meaning. A model turns the query and each document into embeddings, and the system returns the documents whose vectors are closest. It handles synonyms and paraphrases well. But it can blur exact details. A query for ERR_4021 may bring back pages about ERR_4012 because the two look alike to the model.

Keyword, vector, and hybrid search on two example queries A grid with two example queries and three methods. For the exact code query ERR_4021, keyword search finds the right document, vector search may miss it, and hybrid search finds it. For the paraphrased query "laptop will not turn on", keyword search misses the document titled "Device fails to boot", while vector search and hybrid search find it. QUERY KEYWORD VECTOR HYBRID ERR_4021 Target: "Fix for error ERR_4021" Exact code, no paraphrase Finds it exact token match May miss it similar codes look close Finds it keyword list rescues it laptop will not turn on Target: "Device fails to boot" Same meaning, no shared words Misses it no words in common Finds it meaning is close Finds it vector list rescues it
Each method fails on a different kind of query, and hybrid search covers both. The cases are illustrative, not measured results.

A hybrid system needs two indexes over the same documents.

The keyword side usually scores documents with BM25 (Best Matching 25). BM25 rewards a document when:

  • it contains the query terms, and more often is better, with diminishing returns
  • the terms are rare across all documents, so they carry more information
  • the document is not much longer than average, so long pages do not win just by size

The vector side embeds the query with the same model used for the documents, then ranks by a distance measure such as cosine distance. See vector representations for how text becomes vectors and vector distance metrics for how closeness is measured. Large collections use an approximate index, such as DiskANN or HNSW, so the search stays fast.

Merging the lists with Reciprocal Rank Fusion

Now there are two ranked lists. The obvious move is to add the scores, but that does not work. BM25 scores have no fixed upper limit, while vector similarity or distance lives on a different scale. One side would swamp the other.

Reciprocal Rank Fusion (RRF) avoids the problem by ignoring raw scores and using only rank positions. Each document earns a small amount from every list it appears in:

RRF score = sum over lists of  1 / (k + rank)

Here rank is the document’s position in a list (1 is best) and k is a smoothing constant, commonly 60. A document ranked high in both lists gets the highest total. A document in only one list still gets a partial score.

Take four documents, A to D, with k = 60:

  • The keyword list ranks A first, C second, B third. D is absent.
  • The vector list ranks B first, A second, D third. C is absent.
Reciprocal Rank Fusion worked example Two ranked lists feed a fused ranking. The keyword list is A, C, B. The vector list is B, A, D. With k equal to 60 and equal weights, the fused scores are A 0.0325, B 0.0323, C 0.0161, and D 0.0159, so the final order is A, B, C, D. Keyword list (BM25) Vector list Fused ranking (k = 60, equal weights) 1. A 1/61 2. C 1/62 3. B 1/63 D not in this list 1. B 1/61 2. A 1/62 3. D 1/63 C not in this list 1. A 0.0325 1/61 + 1/62 2. B 0.0323 1/63 + 1/61 3. C 0.0161 1/62 only 4. D 0.0159 1/63 only Scores are illustrative. Appearing high in both lists beats appearing high in only one.
RRF adds 1 / (k + rank) from each list, so documents that rank well in both lists rise to the top.

A and B appear in both lists, so they beat C and D, which appear in only one. A narrowly wins because it is first in one list and second in the other.

Weights

You can make one retriever count more. A weight multiplies a list’s contribution. If the vector list is weighted 2 and the keyword list 1, each vector term becomes 2 / (k + rank). In the example, B (first in the vector list) then scores about 0.04866 and edges past A at about 0.04865. D, which only the vector list found, overtakes C. Final order: B, A, D, C.

Use higher vector weight when queries are conversational. Use higher keyword weight when users search for exact names, codes, or identifiers.

Hybrid search in practice: Azure Cosmos DB

Many engines support hybrid search, including Azure AI Search, Elasticsearch, OpenSearch, Weaviate, and PostgreSQL with pgvector plus full-text search. Azure Cosmos DB for NoSQL is a clear example because a document can hold its text and its vector side by side.

Hybrid search pipeline A query splits into two branches. The top branch scores documents with BM25 over a full-text index and produces a ranked list. The bottom branch embeds the query and searches a vector index by distance, producing a second ranked list. Both lists enter Reciprocal Rank Fusion, which outputs one fused top-k result list. Query text from user Full-text search BM25 keyword scoring Vector search embed query + distance Ranked list keyword ranks Ranked list vector ranks RRF rank fusion Top k one ranking
Two retrievers run in parallel and RRF turns their two ranked lists into a single top-k result.

Setup has four steps:

  1. Enable the vector indexing and search feature on the account.
  2. Create a container with a vector policy, a full-text policy, a vector index, and a full-text index.
  3. Insert documents that have both a text property and a vector property.
  4. Run hybrid queries.

A vector policy describes the embedding field. The dimensions must match your embedding model, and the distance function should match the metric that model was trained for:

{
  "vectorEmbeddings": [
    {
      "path": "/vector",
      "dataType": "float32",
      "distanceFunction": "cosine",
      "dimensions": 1536
    }
  ]
}

The indexing policy then adds a full-text index on the text path and a DiskANN vector index on the vector path. Excluding the vector path from the regular index keeps writes cheaper.

A hybrid query calls RRF inside ORDER BY RANK, combining a VectorDistance score and a FullTextScore score:

SELECT TOP 10 *
FROM c
ORDER BY RANK RRF(
  VectorDistance(c.vector, @queryVector),
  FullTextScore(c.text, "reset", "password"),
  [2, 1]
)

The last argument is the optional weights array. Here the vector score counts twice as much as the BM25 score. Your application must embed the user’s question first and pass the result as @queryVector.

One limitation to plan for: vector policies and vector indexes cannot be changed after the container is created. Choose the dimensions and distance function carefully, or you will need a new container. Check the current Azure Cosmos DB hybrid search documentation for the latest syntax and limits.

Where hybrid search fits in RAG

In a RAG system, retrieval quality limits answer quality. If the right document is not in the top results, the model cannot use it. Hybrid search raises the chance that the right passages arrive. Those passages then fill the model’s context window, so how many you return also matters for context engineering.

Some teams add a reranker after fusion. A reranker is a slower model that reads the query and each candidate together and reorders them. Hybrid search fetches a broad, good candidate set cheaply, and the reranker polishes the top of it.

Common pitfalls

  • Adding raw scores. BM25 and vector scores are not on the same scale. Fuse by rank, as RRF does, or normalize carefully.
  • Mismatched embeddings. The query and documents must use the same embedding model, and the distance metric should match what that model expects.
  • Ignoring text analysis. Language, stemming, and stop-word settings change keyword results. Set the language of the full-text index to match your content.
  • Guessing weights. Start with equal weights, then tune on a set of real queries with known good answers.
  • Assuming hybrid always wins. It usually helps, but it adds an index, storage, and latency. Measure it against keyword-only and vector-only search on your own data.

Saved in this browser only. Not synced to your account.