Building the Prompt with Context

Example

Chunks arrive from the retriever with scores. We drop weak ones, keep the rest within a token budget, then build the prompt:

retrieved = [
    {"id": 1, "source": "refund-policy.md", "score": 0.83, "text": "Refunds reach the original payment method in 5-7 working days."},
    {"id": 2, "source": "refund-policy.md", "score": 0.79, "text": "COD refunds go to your bank account or ShopEase wallet."},
    {"id": 3, "source": "delivery.md",      "score": 0.41, "text": "Orders above Rs 499 ship free in metro cities."},
]

def build_prompt(question, chunks, budget_tokens=60, min_score=0.5):
    used, docs = 0, []
    for c in sorted(chunks, key=lambda c: -c["score"]):
        cost = len(c["text"]) // 4 + 10            # rough token estimate + tag overhead
        if c["score"] < min_score or used + cost > budget_tokens:
            continue
        used += cost
        docs.append(f'<document id="{c["id"]}" source="{c["source"]}">\n{c["text"]}\n</document>')
    return ("Answer the question using only the documents. Cite document ids like [1]. "
            "If the documents do not contain the answer, say you don't know.\n\n"
            "<documents>\n" + "\n".join(docs) + "\n</documents>\n\n"
            f"<question>{question}</question>"), used

prompt, used = build_prompt("I paid cash on delivery. How do I get my refund?", retrieved)
print(prompt)
print(f"\n~{used} tokens of context")

Output

Answer the question using only the documents. Cite document ids like [1]. If the documents do not contain the answer, say you don't know.

<documents>
<document id="1" source="refund-policy.md">
Refunds reach the original payment method in 5-7 working days.
</document>
<document id="2" source="refund-policy.md">
COD refunds go to your bank account or ShopEase wallet.
</document>
</documents>

<question>I paid cash on delivery. How do I get my refund?</question>

~48 tokens of context

A Good RAG Prompt

Assembling the Prompt

Example