Human review is an operating design
Adding a person after a model does not create meaningful oversight. A reviewer must receive the relevant evidence, understand the system’s limitations, have enough time and competence to assess the case, be free to disagree, and possess authority to correct, escalate, reverse, or stop the action.
Choose the right oversight mode
| Mode | Appropriate use | Required safeguard |
|---|---|---|
| Human decision | Rights-, safety-, livelihood-, or materially consequential outcomes | The system may support research or drafting, but a qualified person makes and records the decision. |
| Human in the loop | External, financial, privileged, or difficult-to-reverse action | Execution pauses for approval of the exact action, parameters, evidence, impact, and recovery method. |
| Human on the loop | Low-risk, observable, reversible, pre-authorized operation | The operator sees live state, exceptions, budget, errors, and a stop control that disables execution. |
| Post-action sampling | High-volume, low-consequence work with strong deterministic controls | Representative sampling, rapid correction, incident thresholds, and automatic withholding of exceptions. |
Design the approval surface for a real decision
- Plain-language action and exact structured parameters.
- Before-and-after state or proposed external message.
- Source evidence, validation results, conflicts, and missing information.
- People, systems, records, money, and recipients affected.
- Risk tier, triggered policy, alternatives, and the option to reject or edit.
- Expiration, cumulative budget, and rollback or compensation method.
Watch for approval theater
Very high acceptance rates can mean excellent system performance, but they can also reveal automation bias, time pressure, weak interfaces, or incentives that punish disagreement. Review quality must be tested with known-error cases, reviewer agreement, override rationale, queue age, and interviews about whether people can actually stop the process.
Operational measures
Review burden
Minutes per case, queue age, workload, after-hours demand, and exception concentration.
Decision quality
Acceptance, correction, override, escalation, appeal, and reversal outcomes.
Control effectiveness
Actions blocked, approvals expired, unauthorized attempts, stop events, and rollback success.
Human factors
Training, comprehension, confidence to disagree, fatigue, accessibility, and perceived pressure.
When human review is not enough
A person cannot compensate for unrestricted credentials, hidden side effects, missing logs, poor data rights, impossible workloads, or an irreversible action that occurs before review. Technical boundaries and deterministic enforcement must reduce the decision to something a human can responsibly approve.
