Human-in-the-loop is not a workaround for immature technology — it's a deliberate design pattern for production systems. The goal is calibrated autonomy: granting the agent full autonomy for high-confidence, reversible, low-stakes actions while routing uncertain or high-risk actions through human approval. Gartner reports the average cost of an unauthorized AI action incident at $2.4 million (2025), and 78% of organizations now use AI agents with external action capabilities. The EU AI Act mandates human oversight for high-risk AI decisions effective August 2026. Designing the human-AI boundary isn't optional — it's a legal and financial requirement.

Confidence thresholds define the boundary between autonomous action and human escalation. Research shows optimal thresholds vary significantly by domain: general operations at 50–70%, customer service at 80–85%, financial services at 90–95%, and healthcare at 95%+. The "70–80% rule" for customer service indicates that businesses handle 70–80% of routine queries autonomously while escalating the remaining 20–30% to human specialists. Sustainable escalation rates are 10–15% — higher than that overwhelms review teams and negates the efficiency gains. Setting thresholds too low creates risk; setting them too high creates a system that escalates everything and provides no value.

Beyond confidence scores, five categories of signals should trigger human escalation. Explicit request: the user directly asks to speak with a human — never argue with this. Sentiment detection: frustration, anger, or distress detected in the conversation — the agent should recognize emotional states it can't handle. Repeated failure: after two failed attempts to resolve, escalate rather than trying a third time. High-value/high-risk: financial approvals above thresholds, compliance questions, legal matters, PII modifications. Novel scenarios: situations the agent hasn't encountered before, where no retrieval matches exist and the agent would be guessing. Each trigger should be independently monitored and logged.

ESCALATE.md is an open standard (v1.0, 2026) that defines human approval protocols for AI agents. Organizations place this plain-text file in their repository root to specify which actions require human sign-off: production deployments, external communications, financial transactions, data deletion. The standard defines three core components: triggers (which actions always require approval), channels (notification methods — Slack, email, PagerDuty — with 10–15 minute timeouts), and approval handling (30-minute default timeout, denial halts and logs, approval proceeds and logs). This codifies the human-AI boundary as infrastructure, not an afterthought.

The worst human-in-the-loop design turns humans into validators — clicking "approve" on a stream of agent decisions without meaningful review. This creates the illusion of oversight while providing none. Effective workflows position humans as decision-makers applying judgment in three scenarios: evolving contexts the agent can't fully model (a customer's tone suggests something the words don't say), ambiguous situations requiring judgment calls (two valid interpretations of a contract clause), and high-stakes decisions with irreversible consequences (terminating a vendor relationship). The agent should present its analysis, reasoning, and recommendation — but the human makes the call.

The most frustrating experience for an enterprise user is being escalated from an AI agent to a human — and having to repeat everything. Effective handoff design requires the agent to pass a complete context package to the human: the original request, what the agent tried, what it found, why it's escalating, and what it recommends. The human should be able to pick up exactly where the agent left off. This requires structured handoff schemas — not just a chat transcript, but a structured summary with the key entities, decisions made, decisions pending, and relevant documents already retrieved. The handoff is a product feature, not a failure mode.

Every time a human overrides, corrects, or approves an agent decision, that's training data. The most valuable human-AI workflows capture this feedback systematically: what did the agent propose? What did the human decide? Why was it different? Over time, this feedback loop allows the organization to lower confidence thresholds on decisions where the agent consistently agrees with humans, and raise thresholds on decisions where disagreement is frequent. The feedback also reveals patterns: if the agent is consistently wrong about a specific document type or customer segment, that's a signal to improve the underlying model, retrieval, or tool configuration — not just to add more human reviewers.

The review interface is where human-AI workflow design succeeds or fails. A simple "Approve / Reject" button with a wall of text creates automation bias — humans approve 95%+ without meaningful review. Effective review interfaces force engagement: highlight what changed (diff view against the original), surface the agent's uncertainty (which fields had low confidence?), require active selection (choose between options, not just approve/reject), and time the review (reviews under 3 seconds are likely rubber stamps). The interface should make it easier to do the right thing than to click through. This is UX design applied to oversight — and it's as important as the agent's accuracy.