Real World Example

Where Each Approach Is Used

Khan Academy (Khanmigo), Duolingo: mostly careful prompting, examples and evals on top of large hosted models, so they can switch models as better ones arrive.
Banks and insurers: fine-tune small open models for intent detection and run them on their own servers for data-control reasons.
Indian language startups (Sarvam AI and others): train and fine-tune models for Indic languages where general models are weaker.
E-commerce: distil a large model's product-attribute extraction into a small model that processes millions of listings cheaply.

ShopEase Ticket Router: Cost Comparison

Example

ShopEase gets 3 lakh tickets a day (about 300 input + 5 output tokens each). Compare a large model, a small hosted model, and a self-hosted fine-tuned model. Prices are per 1M tokens (input/output) and change; check the pricing page.

tickets_per_day = 300_000
in_tok, out_tok = 300, 5
PRICES = {"claude-opus-5-5": (4.00, 20.00), "claude-haiku-5-5": (0.10, 0.50)}

for model, (pin, pout) in PRICES.items():
    per_day = tickets_per_day * (in_tok * pin + out_tok * pout) / 1_000_000
    print(f"{model:18} USD {per_day:8.2f}/day  USD {per_day * 30:9.2f}/month")

gpu_per_hour = 1.20       # example price of one cloud GPU running the fine-tuned 1.5B model
print(f"{'fine-tuned 1.5B':18} USD {gpu_per_hour * 24:8.2f}/day  USD {gpu_per_hour * 24 * 30:9.2f}/month (+ team time)")

Output

claude-opus-5-5    USD   390.00/day  USD  11700.00/month
claude-haiku-5-5   USD     9.75/day  USD    292.50/month
fine-tuned 1.5B    USD    28.80/day  USD    864.00/month (+ team time)

Reading the Numbers

At 3 lakh tickets a day, the large model costs about USD 390 a day, roughly 40 times the small hosted model's USD 9.75. The self-hosted fine-tuned model looks cheap per token, but its GPU runs 24 hours whether tickets arrive or not, so at this volume it costs about three times more than Haiku, before counting the engineers who build the training pipeline, monitor drift and retrain. Self-hosting wins only when volume is high enough to keep the GPU busy, or when rules require data to stay on your own servers.

Notice the pattern in the companies above: those that need flexibility (education apps switching to newer models) stay with prompting; those with huge, narrow, repetitive workloads or strict data rules (banks, catalogue processing) fine-tune or distil.

Decision

For ShopEase, Haiku with a good few-shot prompt is already cheap and needs no training pipeline, so it is the first choice. Fine-tuning becomes attractive only at much higher volume, with strict on-premise rules, or if evals show the small hosted model is not accurate enough.

Compare total cost (tokens, GPUs, people), not just price per token.
Hosted small models are often the cheapest path for narrow tasks.
Fine-tune or self-host for very high volume, data residency, or a quality gap shown by evals.