A pilot can prove output and still fail the operating decision
Demonstrations are optimized to show that a capability can work. Production requires evidence that the complete workflow creates value repeatedly, stays within quality and risk limits, has an owner, can be monitored and supported, and can recover from failure or change.
The missing production contract
| Production question | Required evidence |
|---|---|
| Who owns the business result? | Named process owner, operating metric, benefit mechanism, and authority to change the workflow. |
| What exactly is being released? | Versioned model, prompts, rules, data sources, tools, integrations, permissions, interface, and environment. |
| How good is good enough? | Frozen acceptance set, primary threshold, critical-error limits, subgroup or scenario results, and human review. |
| How will it operate? | Service owner, runbook, monitoring, alerting, incident process, support hours, capacity, cost, and escalation. |
| How will it fail safely? | Fallback process, feature flag, credential revocation, rollback, data correction, and customer recourse. |
| What will change the decision? | Model, data, policy, workflow, vendor, population, incident, drift, or cost triggers for reassessment. |
Common scale blockers
- The pilot population was curated and does not represent normal exceptions or difficult cases.
- The model improved an offline metric but did not change the end-to-end process outcome.
- Human review, error correction, and exception handling destroy the expected economics.
- Security, privacy, procurement, or identity requirements were postponed until after the demo.
- No service owner accepted monitoring, incident, change, and decommissioning obligations.
- A vendor or model change invalidates the original test without a regression asset.
Use three scale outcomes, not a forced success narrative
| Outcome | Meaning |
|---|---|
| Scale | Value, quality, security, operations, ownership, and economics pass; production conditions and funding are committed. |
| Conditional scale | Hard security and authority gates pass, but a bounded quality, source, adoption, or operating issue has a dated remediation plan and limited exposure. |
| Do not scale | The value case fails, authority or controls cannot be established, operating cost is unjustified, or the need is better solved by a simpler intervention. |
Production readiness is renewable
Approval should expire after a defined period or material change. Continued use depends on ongoing value, current documentation, functioning controls, acceptable incidents, valid data, supported components, and evidence that people can still operate and challenge the system.
