Real World Example · Sampling and Generation Settings in Generative AI | Generative AI & LLMs

Real World Example

Settings per Task at a Food Delivery App

A Zomato-style app uses AI in several places, each with different needs:

Feature	Need	Open model (Ollama/vLLM) settings	Claude settings
Tag a complaint	Same answer every time	temperature 0, max_tokens 5	Haiku, effort low, fixed tag list in prompt
Reply to a review	Friendly, slightly varied	temperature 0.7, top_p 0.9	Sonnet, tone examples in prompt
Push notification ideas	Creative	temperature 1.1, top_p 0.95	Sonnet, "give 10 varied options"

Getting Variety Without Temperature

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=400,
    system="You write short push notifications for a food delivery app in India. "
           "Each under 60 characters. Use Hinglish where natural. No emojis.",
    messages=[{"role": "user", "content": "Give 5 very different notification ideas for a rainy evening."}],
)
print(response.content[0].text)

Run it

python notify.py

Example output

1. Baarish + pakode = perfect evening. Order now
2. Rainy day? Hot chai is 20 mins away
3. Stay dry. We'll bring the biryani to you
4. Monsoon special: 30% off on soups tonight
5. Cancel the cooking plan. It's raining