Careeroza
Context Window
The Model's Working Memory
The context window is the maximum number of tokens the model can handle in one request: system prompt + all messages + documents you add + the reply it writes. Current Claude models have a 1M-token window; many open models have 8k–128k. Anything outside the window is invisible to the model.
┌──────────────────────── context window ────────────────────────┐ │ system prompt │ chat history │ retrieved docs │ user question │ reply │ └────────────────────────────────────────────────────────────────┘ input tokens ─────────────────────────────────────────────► output tokens
- The model has no memory between API calls. Your app must resend the history each time.
- A bigger window is not free: more input tokens = more cost and slower replies. Send only what is useful.
Will It Fit?
Example
Before sending a request, add up the parts and keep room for the reply (max_tokens). Here a small 8,000-token open model is assumed:
WINDOW = 8_000
MAX_OUTPUT = 1_000
def tokens(text):
return max(1, round(len(text) / 4)) # estimate
system = "You are the ShopEase support assistant. Be brief and polite."
history = ["Hi, my order is late." , "Sorry! Can you share the order ID?"] * 40
docs = ["Delivery policy: orders ship in 2 days and arrive in 3-5 days." * 30]
question = "It is SE-10423. When will it arrive?"
parts = {
"system": tokens(system),
"history": sum(tokens(m) for m in history),
"docs": sum(tokens(d) for d in docs),
"question": tokens(question),
}
used = sum(parts.values())
for k, v in parts.items():
print(f"{k:9} {v:6}")
print(f"{'input':9} {used:6} + reserved output {MAX_OUTPUT}")
print("fits:", used + MAX_OUTPUT <= WINDOW, f"(window {WINDOW})")Output
system 15 history 520 docs 465 question 9 input 1009 + reserved output 1000 fits: True (window 8000)
Tip: When history is too long, drop the oldest turns or replace them with a short summary. The medium level's chatbot module shows both.