Careeroza

Handling Errors

What Can Go Wrong in a Chatbot

ProblemWhat the user seesWhat the code does
429 rate limit / 529 overloaded"Busy, retrying..." then the answerSDK retries with backoff; then fall back to another model
Timeout or network drop mid-streamPartial answer + "Connection lost, tap to retry"Do not save the partial reply to history
stop_reason == "max_tokens"Answer ends with "..." and a "continue" optionRaise max_tokens or ask for shorter answers
stop_reason == "refusal"A polite fixed messageLog it; offer a human agent
400 bad request (bug)"Something went wrong"Alert developers; never retry

Fallbacks with a Scripted Client

Example

Try the main model, fall back to a second one on overload, and always end with a friendly message instead of a crash. The scripted_call function stands in for the API:

class Overloaded(Exception): pass
class BadRequest(Exception): pass

SCRIPT = {"claude-sonnet-5-5": Overloaded("529 overloaded_error"),
          "claude-haiku-5-5": "We deliver on Sundays in most metro cities."}

def scripted_call(model, message):
    result = SCRIPT[model]
    if isinstance(result, Exception):
        raise result
    return result

def reply(message, models=("claude-sonnet-5-5", "claude-haiku-5-5")):
    for model in models:
        try:
            text = scripted_call(model, message)
            return {"model": model, "text": text}
        except Overloaded as e:
            print(f"{model}: {e} -> trying next model")
        except BadRequest as e:
            print(f"{model}: {e} -> bug, not retrying")
            break
    return {"model": None, "text": "Sorry, our assistant is busy. A human agent will reply shortly."}

print(reply("Do you deliver on Sundays?"))
SCRIPT["claude-haiku-5-5"] = Overloaded("529 overloaded_error")
print(reply("Do you deliver on Sundays?"))

Output

claude-sonnet-5-5: 529 overloaded_error -> trying next model
{'model': 'claude-haiku-5-5', 'text': 'We deliver on Sundays in most metro cities.'}
claude-sonnet-5-5: 529 overloaded_error -> trying next model
claude-haiku-5-5: 529 overloaded_error -> trying next model
{'model': None, 'text': 'Sorry, our assistant is busy. A human agent will reply shortly.'}