Careeroza

Counting Tokens

Use the Real Counter

Every model family has its own tokenizer, so the same text gives different counts. For Claude, use the token counting endpoint client.messages.count_tokens(...) with the same model you will call. Do not use tiktoken (that is OpenAI's tokenizer; it undercounts Claude tokens).

import anthropic

client = anthropic.Anthropic()

count = client.messages.count_tokens(
    model="claude-opus-5-5",
    system="You are the ShopEase support assistant.",
    messages=[{"role": "user", "content": "Where is my order SE-10423? It was due yesterday."}],
)
print("input tokens:", count.input_tokens)

Run it

python count.py

Example output

input tokens: 31

A Quick Estimator

Example

For back-of-the-envelope planning (before you have an API key), a characters-divided-by-4 estimate is good enough for English. Always confirm with the real counter before a launch:

def estimate_tokens(text):
    return max(1, round(len(text) / 4))

samples = [
    "Hi",
    "Where is my order SE-10423? It was due yesterday.",
    "ShopEase return policy: items can be returned within 7 days of delivery " * 5,
]
for s in samples:
    print(f"{len(s):4} chars ~ {estimate_tokens(s):3} tokens | {s[:40]!r}")

Output

   2 chars ~   1 tokens | 'Hi'
  49 chars ~  12 tokens | 'Where is my order SE-10423? It was due y'
 360 chars ~  90 tokens | 'ShopEase return policy: items can be ret'