Checklist Before Going Live
Why a Checklist
A go-live checklist (or launch readiness review) is a written list of conditions that must all be true before real users get access. Pilots and surgeons use checklists for the same reason: experienced people still forget steps under pressure, and the cost of one forgotten step is high. GenAI launches have more such steps than usual web launches, because the risks span quality, safety, privacy, cost and operations at once.
Each item below links back to a module in this level, and each is verifiable: someone can show evidence (an eval report, a config screenshot, a load test result), not just say "yes, we did that". Assign an owner to every item, and treat any unchecked safety or privacy item as a launch blocker. The list is grouped by who usually owns it: product and quality, safety and privacy, and operations.
Product and Quality
- Eval thresholds met; golden set includes Hinglish, typos and edge cases.
- Clear "talk to a human" path, and handover includes the chat summary.
- Users are told they are chatting with an AI assistant.
Safety and Privacy
- PII masked before LLM calls and in logs; log retention set (for example 90 days).
- Tools enforce per-customer access and refund limits in code.
- Injection tests pass; output filter and HTML escaping on.
- DPDP basics reviewed with legal: notice, consent, deletion requests.
Operations
- API keys in Secret Manager, separate per environment, with spend limits and alerts.
- Retries, timeouts, fallback model, and a friendly error message when everything fails.
- Dashboards: latency, errors, cost per route, feedback; on-call owner named.
- Prompt versions in git; CI runs evals; canary deploys with rollback.
- Load test done; rate limits per user (OWASP LLM10).
After Launch and Summary
Going live is the start of a new phase, not the end. Plan a soft launch: a small share of users, or one category of questions, for the first week, with the team reading a sample of conversations daily. Agree in advance what would trigger a rollback (for example, any wrong refund, or thumbs-down rate above a set level), and make sure turning the assistant off is a single switch that sends users to the old support flow.
Note: Common gaps found in reviews: no kill switch, no per-user spend limit, logs containing raw phone numbers, a fallback model that was never evaluated, and nobody named on call for the first weekend. Each is cheap to fix before launch and expensive after.
- A checklist turns "we think it is ready" into verifiable evidence with owners.
- Cover quality, safety and privacy, and operations; safety items are blockers.
- Soft launch with daily review, clear rollback triggers and a kill switch.