Chunking Strategies

Why Chunk?

Documents are split into chunks (typically 200–800 tokens) before embedding. One vector for a 30-page PDF would blur all topics together; small focused chunks match questions precisely. Overlap between chunks keeps a sentence that crosses a boundary in at least one chunk.

Strategy	How	Good for
Fixed size + overlap	Every N words/tokens, with M overlapping	Plain text, quick start
Structure-based	Split on headings, paragraphs, list items	Policies, docs, Markdown, HTML
Sentence-based	Group whole sentences up to a size	Articles, chats
Semantic	Split where the topic changes (embedding distance)	Long mixed documents

Fixed Size with Overlap

Example

Words stand in for tokens here. Each chunk has 20 words and repeats the last 5 words of the previous one:

text = ("ShopEase accepts returns within 7 days of delivery. Items must be unused with tags. "
        "Refunds are sent to the original payment method within 5-7 working days. "
        "Cash on delivery orders are refunded to a bank account or ShopEase wallet. "
        "Electronics have a 10-day replacement-only policy.")

def chunk_words(text, size=20, overlap=5):
    words = text.split()
    step = size - overlap
    chunks = []
    for start in range(0, len(words), step):
        chunks.append(" ".join(words[start:start + size]))
        if start + size >= len(words):
            break
    return chunks

for i, c in enumerate(chunk_words(text)):
    print(f"[{i}] ({len(c.split())} words) {c}")

Output

[0] (20 words) ShopEase accepts returns within 7 days of delivery. Items must be unused with tags. Refunds are sent to the original
[1] (20 words) are sent to the original payment method within 5-7 working days. Cash on delivery orders are refunded to a bank
[2] (15 words) are refunded to a bank account or ShopEase wallet. Electronics have a 10-day replacement-only policy.

Structure-based Chunking

Example

For Markdown or HTML policies, splitting on headings keeps each rule together, and the heading becomes useful metadata:

doc = """# Returns
Returns are accepted within 7 days. Items must be unused.

# Refunds
Refunds reach the original payment method in 5-7 working days.
COD refunds go to your bank account or wallet.

# Exchanges
Clothes can be exchanged for another size within 10 days."""

chunks, heading, lines = [], None, []
for line in doc.splitlines():
    if line.startswith("# "):
        if heading:
            chunks.append({"section": heading, "text": " ".join(lines)})
        heading, lines = line[2:], []
    elif line.strip():
        lines.append(line.strip())
chunks.append({"section": heading, "text": " ".join(lines)})

for c in chunks:
    print(c)

Output

{'section': 'Returns', 'text': 'Returns are accepted within 7 days. Items must be unused.'}
{'section': 'Refunds', 'text': 'Refunds reach the original payment method in 5-7 working days. COD refunds go to your bank account or wallet.'}
{'section': 'Exchanges', 'text': 'Clothes can be exchanged for another size within 10 days.'}

Tip: Prepend the document title and section to each chunk before embedding ("Refund policy > COD refunds: ..."). Chunks that say "this" or "it" without context retrieve badly.