Hybrid Search and Reciprocal Rank Fusion

Combining Two Rankings

Hybrid search runs keyword search and vector search, then merges the two result lists. Their scores are on different scales (BM25 7.2 vs cosine 0.83), so instead of adding scores we merge by rank with Reciprocal Rank Fusion (RRF):

RRF_score(doc) = sum over each result list of  1 / (k + rank_in_that_list)      (k is usually 60)

A document that appears near the top of both lists wins. A document in only one list still gets some credit.

Why Merge by Rank

The two searches are good at different things. Keyword search finds exact terms; vector search finds meaning ("earbuds" is close to "electronics"). A real question often needs both, so hybrid search gets better recall (more of the right documents found) than either alone.

The hard part is combining them. A BM25 score of 7.2 and a cosine similarity of 0.83 are not on the same scale, and their ranges change from query to query, so adding them is meaningless. Ranks avoid this: "first place" means the same thing in both lists. It is like combining two judges at a cooking contest who score out of 10 and out of 100: comparing their rankings is fairer than adding raw marks.

Why 1 / (k + rank), and why k = 60? The reciprocal makes rank 1 worth more than rank 10, but with a large k the difference is gentle: 1/61 ≈ 0.0164 for rank 1 and 1/70 ≈ 0.0143 for rank 10. So one list cannot dominate just by putting a document first, and agreement between lists matters most. A smaller k (say 10) trusts top ranks more strongly. The value 60 came from the original RRF paper's experiments and works well as a default.

Example

RRF over two ranked lists for the query "can I return earbuds bought on COD?". The lists are the kind of output a BM25 search and a vector search return.

def rrf(result_lists, k=60):
    scores = {}
    for results in result_lists:
        for rank, doc_id in enumerate(results, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1 / (k + rank)
    return sorted(scores.items(), key=lambda kv: (-kv[1], kv[0]))

bm25_results   = ["cod", "returns", "refunds", "shipping"]       # exact words: COD, return
vector_results = ["returns", "warranty", "refunds", "cod"]       # meaning: earbuds ~ electronics

for doc, score in rrf([bm25_results, vector_results]):
    print(f"{doc:9} {score:.5f}")

Output

returns   0.03252
cod       0.03202
refunds   0.03175
warranty  0.01613
shipping  0.01562

returns is 2nd in BM25 and 1st in vectors: 1/62 + 1/61 ≈ 0.03252, the top score. cod is 1st in BM25 but only 4th in vectors, so it comes second. warranty and shipping appear in only one list each and get roughly half the score. Agreement wins.

Weighted Variant

Some teams weight the lists (for example 0.7 vector, 0.3 keyword) or use a small k to trust top ranks more. Tune these on your eval set, not by feel.

Note: Trade-offs: hybrid search runs two queries, so it costs a little more latency and infrastructure. RRF also throws away score information: if vector search is very confident about one document, RRF does not know. Fusion improves the candidate list; it does not make the final order perfect, which is the job of re-ranking (next page).

Hybrid search = keyword + vector search, merged.
Scores are on different scales, so merge by rank with RRF: sum of 1 / (k + rank).
Documents ranked well by both lists rise to the top; k = 60 is a solid default.
Weights and k are tuned on an eval set.

Exercise

Exercise: Change rrf so each list has a weight, and run it with weights 0.3 for BM25 and 0.7 for vectors. Which document moves?

Show answer

def weighted_rrf(result_lists, weights, k=60):
    scores = {}
    for results, w in zip(result_lists, weights):
        for rank, doc_id in enumerate(results, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + w / (k + rank)
    return sorted(scores.items(), key=lambda kv: (-kv[1], kv[0]))

print(weighted_rrf([bm25_results, vector_results], [0.3, 0.7]))
# "refunds" (rank 3 in both lists) now edges above "cod": cod's high rank came from the down-weighted BM25 list.