RAG Hybrid Search

RAG hybrid search combines keyword search with vector search. Embeddings are great at meaning but weak at exact tokens such as product codes, error numbers, names and rare acronyms. Keyword search like BM25 is the opposite. Using both catches what either would miss.

BM25 Keyword Search

BM25 is the classic ranking function used by search engines. It rewards documents that contain the query words, especially rare ones. Install pip install rank-bm25:

from rank_bm25 import BM25Okapi

docs = [
    "Error E1042 means the printer tray is empty.",
    "The printer shows a paper jam warning.",
    "Restart the router to fix connection drops.",
]
bm25 = BM25Okapi([d.lower().split() for d in docs])
scores = bm25.get_scores("e1042".split())
print([round(s, 2) for s in scores])

Sample output:
[0.98, 0.0, 0.0]

A vector search for "E1042" might return the paper jam document because both are about printers. BM25 nails the exact code.

Reciprocal Rank Fusion (RRF)

Keyword and vector scores are on different scales, so instead of adding scores we combine rankings. RRF gives each document 1 / (k + rank) from every list and sums them:

def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

keyword_rank = ["d1", "d3", "d2"]
vector_rank  = ["d2", "d1", "d4"]
print(rrf([keyword_rank, vector_rank]))

['d1', 'd2', 'd3', 'd4']

d1 is near the top of both lists, so it wins. RRF is simple, needs no tuning and works surprisingly well.

Hybrid Search in Vector Databases

• Qdrant: store dense and sparse (BM25/SPLADE) vectors and fuse them in one query.
• pgvector: combine PostgreSQL full-text search (tsvector) with vector ordering in SQL.
• Weaviate, Elasticsearch and OpenSearch: built-in hybrid queries.
• LangChain: EnsembleRetriever combines retrievers with weighted RRF.

When Hybrid Search Helps Most

Use it for technical docs, product catalogues, legal text with section numbers, support tickets with error codes, and any content full of names. For casual conversational content, pure vector search is often enough. As always, measure on real queries.

Frequently asked questions

What is hybrid search in RAG?

Running keyword (sparse) search and vector (dense) search together and merging the results, usually with Reciprocal Rank Fusion.

Is BM25 better than embeddings?

Neither is better overall. BM25 wins on exact terms and rare words; embeddings win on paraphrases and meaning. Hybrid usually beats both.

What value of k should I use in RRF?

k=60 is the common default from the original paper and rarely needs changing.