Careeroza

Why Structured Output

Code Needs Data, Not Prose

When code consumes the model's answer (save to a database, call an API, show in a UI), you need structured output: JSON with known fields and types. Asking nicely ("reply in JSON") usually works, but "usually" breaks production: an extra sentence before the JSON, a missing field, a string instead of a number.

What Goes Wrong with Free-form Replies

Example

Three stand-in replies to "extract the order ID and rating as JSON". Only one parses cleanly:

import json

replies = [
    '{"order_id": "SE-4410", "rating": 4}',
    'Sure! Here is the JSON:\n{"order_id": "SE-4410", "rating": 4}',
    '{"order_id": "SE-4410", "rating": "four"}',
]
for r in replies:
    try:
        data = json.loads(r)
        ok = isinstance(data.get("rating"), int)
        print("parsed  ", data, "| rating is int:", ok)
    except json.JSONDecodeError as e:
        print("FAILED  ", repr(r[:30]), "->", e.msg)

Output

parsed   {'order_id': 'SE-4410', 'rating': 4} | rating is int: True
FAILED   'Sure! Here is the JSON:\n{"orde' -> Expecting value
parsed   {'order_id': 'SE-4410', 'rating': 'four'} | rating is int: False

Two fixes, both covered next: (1) let the API enforce a JSON schema so the reply always matches; (2) validate in your code anyway and retry or fall back when something is off.