Query Rewriting and HyDE

Fix the Query Before Searching

Users write short, messy, context-dependent questions: "and for COD?", "it didnt come, now what". Searching with that text gives bad results. Ask a cheap model to improve the query first.

Technique	What the model does
Rewrite to standalone	uses chat history: "and for COD?" → "What is the refund time for cash on delivery orders?"
Multi-query	writes 3-4 different phrasings, search each, merge with RRF
HyDE	writes a hypothetical answer, and you search with that answer (answers look like documents, questions do not)

Example

HyDE. fake_llm is a stand-in for the model writing a hypothetical answer. The raw question shares almost no words with the right document; the hypothetical answer does.

import re
STOP = {"my", "for", "the", "a", "to", "of", "is", "are", "when", "will", "i", "get", "in", "after", "within"}
def terms(t): return {w for w in re.findall(r"[a-z]+", t.lower()) if w not in STOP}

DOCS = {
    "refunds":  "Refunds are credited to the original payment method within 5 to 7 working days after pickup.",
    "shoes":    "Shoes can be exchanged for another size within 10 days.",
    "delivery": "Standard delivery takes 3 to 5 days.",
}
def search(text):
    scores = {d: len(terms(text) & terms(body)) for d, body in DOCS.items()}
    return sorted(scores.items(), key=lambda kv: (-kv[1], kv[0]))

def fake_llm(prompt):  # stand-in for the model
    return ("Refunds for returned items are credited to your original payment method "
            "a few working days after the pickup is done.")

question = "when will I get my money for the returned shoes?"
print("raw question :", search(question))
hypo = fake_llm(f"Write a short passage that answers: {question}")
print("HyDE passage :", search(hypo))

Output

raw question : [('shoes', 1), ('delivery', 0), ('refunds', 0)]
HyDE passage : [('refunds', 8), ('delivery', 1), ('shoes', 1)]

Rewriting with Claude

import anthropic
client = anthropic.Anthropic()

def standalone_query(history: list[dict], question: str) -> str:
    convo = "\n".join(f"{m['role']}: {m['content']}" for m in history[-6:])
    resp = client.messages.create(
        model="claude-haiku-5-5",                    # cheap and fast is enough here
        max_tokens=200,
        output_config={"effort": "low"},
        system="Rewrite the user's last question as one standalone search query. Output only the query.",
        messages=[{"role": "user", "content": f"<history>\n{convo}\n</history>\n<question>{question}</question>"}],
    )
    return next(b.text for b in resp.content if b.type == "text").strip()

Sample output

python rewrite.py

Example output

history : user: How long do refunds take for UPI payments?  assistant: 5-7 working days.
question: and for COD?
rewritten: refund time for cash on delivery (COD) orders