Careeroza

Summarizing Long Documents

Choosing a Strategy

StrategyHowWhen
StuffPut the whole document in one promptFits comfortably in the context window (most documents with 1M-token models)
Map-reduceSummarize each chunk separately (map), then summarize the summaries (reduce)Very long or many documents; parallel and cheaper with a small model
RefineWalk through chunks, updating a running summaryOrder matters (stories, long chats); slow, sequential

Even when everything fits, a focused summary prompt matters: say who the summary is for, how long it should be, and what to include (decisions, numbers, deadlines).

Picking a Strategy by Size

Example

A helper that estimates tokens and picks a strategy. The window and limits are configurable because they depend on the model and on your cost budget:

def estimate_tokens(text):
    return len(text) // 4

def choose_strategy(text, window=1_000_000, max_stuff_tokens=150_000, chunk_tokens=20_000):
    n = estimate_tokens(text)
    if n <= max_stuff_tokens:
        return n, "stuff", 1
    chunks = -(-n // chunk_tokens)            # ceiling division
    return n, "map-reduce", chunks + 1        # one call per chunk + one reduce call

docs = {
    "return policy (2 pages)": "x" * 12_000,
    "seller agreement (40 pages)": "x" * 300_000,
    "annual report (300 pages)": "x" * 2_400_000,
}
for name, text in docs.items():
    n, strategy, calls = choose_strategy(text)
    print(f"{name:28} ~{n:>7,} tokens -> {strategy:10} ({calls} call{'s' if calls > 1 else ''})")

Output

return policy (2 pages)      ~  3,000 tokens -> stuff      (1 call)
seller agreement (40 pages)  ~ 75,000 tokens -> stuff      (1 call)
annual report (300 pages)    ~600,000 tokens -> map-reduce (31 calls)

Tip: Why not always stuff with a 1M window? Very long prompts cost more per call, are slower, and details in the middle can get less attention. Map-reduce with a cheap model is often cheaper and easier to cite.