Chain of Thought

Thinking Before Answering

Chain of thought (CoT) means letting the model work through a problem step by step before giving the final answer. For multi-step problems (calculations, policy rules with conditions, debugging) this improves accuracy, because each generated step becomes context for the next token.

Classic prompting: "Think step by step inside <thinking> tags, then give the final answer inside <answer> tags." Your code shows only the answer to the user.
Built-in thinking: current Claude models think on their own (adaptive thinking). You control how hard with output_config={"effort": ...}; the reasoning comes back as thinking blocks, separate from the text block.

Why Step-by-Step Helps

An LLM writes its answer one token at a time, and each token is predicted from everything already in the context. If the model has to output the final number straight away, all the work must happen inside one prediction. When it first writes the intermediate steps ("subtotal is 3,897", "15% is 584.55, capped at 400"), those steps become part of the context, so the next prediction can build on them instead of guessing. It is like a student solving a maths problem on rough paper: writing each line down means the student does not have to hold everything in their head.

This is why chain of thought helps most on multi-step tasks: arithmetic, applying a policy with several conditions, comparing options, or tracing what a piece of code does. For one-step tasks (translate a sentence, tag sentiment, extract a name) it adds tokens and latency but little accuracy.

How to Use It

Decide if the task needs it. Try the task on 10–20 real examples without reasoning first. If accuracy is already high, stop here.
Pick the mechanism. On current Claude models, prefer built-in thinking and tune effort. With models that do not think on their own, ask for reasoning inside <thinking> tags.
Separate reasoning from the answer. Ask for the final answer in its own tag or block, so your code can show only that part to the customer.
Give the facts, not just the question. Reasoning cannot fix missing data. Put the order date, policy text and prices in the prompt.
Measure. Compare accuracy, cost and latency with and without reasoning on the same test set, then keep the cheapest setting that is accurate enough.

Term	Meaning
Chain of thought (CoT)	The model writes intermediate reasoning steps before the final answer.
Adaptive thinking	Current Claude models decide by themselves how much to think; the reasoning is returned as `thinking` blocks.
Effort	A setting (`output_config={"effort": ...}`) that trades more reasoning for more cost and latency.
Scratchpad	Space (tags or thinking blocks) where reasoning goes, kept apart from the user-facing answer.

Separating Reasoning from the Answer

Example

When you ask for reasoning in tags, your code must pull out only the final answer. The text below is a stand-in for the model's reply:

import re

model_reply = """<thinking>
Order delivered on 10 March. Return window is 7 days, so last day is 17 March.
Customer asked on 15 March, which is within the window.
Item is a phone case: not in the non-returnable list.
</thinking>
<answer>Yes, you can return it. Your return window is open until 17 March.</answer>"""

def extract(tag, text):
    m = re.search(rf"<{tag}>(.*?)</{tag}>", text, re.S)
    return m.group(1).strip() if m else None

print("shown to user:", extract("answer", model_reply))
steps = extract("thinking", model_reply).splitlines()
print("reasoning steps logged:", len(steps))

Output

shown to user: Yes, you can return it. Your return window is open until 17 March.
reasoning steps logged: 3

Built-in Thinking with Effort

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=4000,
    thinking={"type": "adaptive", "display": "summarized"},   # show a summary of the thinking
    output_config={"effort": "high"},                          # low | medium | high | xhigh | max
    messages=[{"role": "user", "content": "A customer bought 3 items at Rs 1,299 with a 15% coupon capped at Rs 400, "
                                          "then returned one item. How much should be refunded?"}],
)
for block in response.content:
    if block.type == "thinking":
        print("[thinking]", block.thinking)
    elif block.type == "text":
        print("[answer]", block.text)

Run it

python refund_math.py

Example output

[thinking] Subtotal 3 x 1299 = 3897. 15% = 584.55, capped at 400, so paid 3497. Discount spread evenly: 400/3 = 133.33 per item. Refund for one item = 1299 - 133.33 = 1165.67.
[answer] Refund Rs 1,165.67 for the returned item (its price Rs 1,299 minus its share of the Rs 400 coupon, Rs 133.33).

Tip: More thinking = more output tokens = more cost and latency. Use low effort for simple tasks and raise it only where accuracy improves on your test set.

Limits and Summary

Note: Common misconceptions. (1) The written reasoning is not a guaranteed, faithful record of how the model reached its answer; it is generated text that usually helps, but it can contain mistakes even when the answer is right, and the other way round. (2) More thinking is not always better: on easy tasks it can add cost and even over-complicate the answer. (3) Reasoning does not create facts. If the return policy is not in the prompt, the model may reason carefully from a wrong assumption.

Chain of thought lets each reasoning step become context for the next token, which helps multi-step problems.
Use built-in adaptive thinking with effort on current Claude models; use tagged reasoning elsewhere.
Show only the final answer to users; log the reasoning for debugging.
Turn it on only where your test set shows a real accuracy gain worth the extra tokens.