Output Filtering

Treat model output like user input: it is untrusted. Before it reaches the user, a browser, a database or another tool, check it.

• Escape HTML before rendering, so a model-written <script> cannot run (XSS).
• Allow only links to your own domains.
• Block promises the business cannot keep ("guaranteed delivery tomorrow", "100% refund").
• Validate actions (JSON) against a schema and business rules before executing.

Input checks try to stop bad requests; output filtering (an output guardrail) checks what the model produced before anything uses it. You need both, because a model can produce harmful output from an innocent input: an injected review, a hallucinated promise, or simply a bad generation. The output is the last point where you are still in control; after it, the text is in a browser, a database or a payment system.

It is the same idea as a newspaper's sub-editor. Reporters can be trusted, yet every article is checked for libel and wrong facts before printing, because the cost of a mistake after printing is much higher. Output checks come in a few kinds:

import html, re
ALLOWED_DOMAINS = {"shopease.in", "help.shopease.in"}
BANNED = [r"\bguarantee(d)?\b", r"\b100% refund\b", r"\blegal advice\b"]

def filter_output(text):
    issues = []
    for url in re.findall(r"https?://([\w.-]+)", text):
        if url not in ALLOWED_DOMAINS:
            issues.append(f"link to {url}")
            text = text.replace(url, "[link removed]")
    for pat in BANNED:
        m = re.search(pat, text, re.IGNORECASE)
        if m:
            issues.append(f"banned phrase '{m.group()}'")
    return html.escape(text), issues

for out in ["Track it at https://shopease.in/orders/SE1042 .",
            "Delivery is guaranteed tomorrow! Pay at http://free-coupons.example",
            "<script>alert('hi')</script> Your refund is processing."]:
    safe, issues = filter_output(out)
    print("issues:", issues or "none")
    print("shown :", safe)

issues: none
shown : Track it at https://shopease.in/orders/SE1042 .
issues: ['link to free-coupons.example', "banned phrase 'guaranteed'"]
shown : Delivery is guaranteed tomorrow! Pay at http://[link removed]
issues: none
shown : <script>alert('hi')</script> Your refund is processing.

The first reply links to ShopEase and passes. The second has two problems: a link to an unknown domain (removed) and the banned word "guaranteed". This filter only reports the phrase; the caller then decides to regenerate or fall back to a safe reply, because deleting words can change the meaning. The third reply's <script> tag is escaped, so the browser shows it as text instead of running it.

from pydantic import BaseModel, Field

class RefundAction(BaseModel):
    order_id: str = Field(pattern=r"^SE\d{4,8}$")
    amount_inr: int = Field(gt=0, le=2000)          # above this needs a human

def execute(action_json: str, customer_orders: set[str]):
    action = RefundAction.model_validate_json(action_json)   # raises if invalid
    if action.order_id not in customer_orders:
        raise PermissionError("order does not belong to this customer")
    return payments.refund(action.order_id, action.amount_inr)

Two separate checks happen here. The schema check (Pydantic) makes sure the action is well-formed: an order ID in the right pattern and an amount between 1 and 2,000. The authorisation check makes sure this customer is allowed to act on this order. A schema-valid action can still be an attack, so never skip the second check.

• Model output is untrusted input to whatever comes next.
• Check format, channel safety, business rules, content and grounding.
• On a failed check: regenerate or fall back; do not silently edit meaning.
• Validate actions for both schema and authorisation before executing.