Human Oversight

Human oversight is the obligation that prevents fully automated, consequential decisions from being made by an AI system without meaningful human review. EU AI Act Article 14 requires that high-risk AI systems be designed and developed so they can be effectively overseen by natural persons during the period in which the system is in use.

The Article distinguishes between oversight measures built into the system itself and measures the deployer must implement. The provider is on the hook for design; the deployer is on the hook for operation.

This obligation also intersects with GDPR Article 22, which gives data subjects the right not to be subject to a decision based solely on automated processing that produces legal or similarly significant effects. In the United States, Colorado SB 24-205 requires deployers of consequential AI to provide human review of adverse decisions.

Vendors solving human oversight typically provide a review-and-approval workflow layered on top of the model's output. The user-facing interface routes high-risk or low-confidence predictions to a human reviewer who can confirm, override, or escalate.

The mature implementations also handle the harder problem: automation bias. If the reviewer just rubber-stamps every model decision, oversight is theatre. Good platforms log reviewer-override rates, alert when override rate drops below a threshold, and require reviewers to record a reason for each decision.

Capabilities to look for:

• Configurable confidence thresholds that route low-confidence predictions to humans.
• Reviewer audit trails with timestamps and reason codes for every override.
• Override-rate dashboards to detect automation bias early.
• Role-based access so different reviewer populations handle different decision types.
• Training-material distribution and reviewer-certification tracking.

• [ ] Define which decisions require human review based on consequence severity, not just model confidence.
• [ ] Train reviewers on the system's known limitations and failure modes.
• [ ] Provide reviewers with the underlying inputs, not just the model's output.
• [ ] Allow reviewers to disregard or override the model's recommendation without penalty.
• [ ] Monitor automation-bias indicators: override rate, review time, agreement rate.
• [ ] Provide a clear escalation path when the reviewer is uncertain.
• [ ] Document the oversight measures in the technical file (Article 11).
• [ ] For GDPR Article 22 cases, ensure the human review is genuine, not pro-forma.

The most common failure: the human reviewer has no real ability to override. The model's recommendation is shown alongside a single "Approve" button with no context about why the model decided what it did. Article 14(4)(a) requires that the reviewer be able to fully understand the relevant capacities and limitations of the high-risk AI system.

The second failure: reviewers approve at rates above 95% because override carries professional risk. The CFPB warned about this exact pattern in their 2023 circular on adverse action notices for AI-driven credit decisions.

The third failure: the system was designed for human oversight, but the deployer turned it off in production for throughput. Article 26 puts that responsibility squarely on the deployer.

• EU AI Act Article 14: Human Oversight — the binding text.
• EU AI Act Article 26: Obligations of Deployers — defines deployer responsibility for operating the oversight measures.
• GDPR Article 22: Automated Decision-Making — the parallel data-protection regime.
• Colorado SB 24-205 — the first US state to require human review of adverse consequential decisions, effective February 1, 2026.
• CFPB Circular 2023-03 on AI in adverse-action notices — US enforcement context for consumer credit.