Human-in-the-Loop
Also known as: HITL, approval gate, interrupt, tool approval, permission prompt, escalation to a human, human as a tool, four-eyes principle
Intent
Stop an agent at a few chosen points, save its full state, and let a person approve, edit, reject or answer before the run continues. Code decides where the run stops, a human decides what happens there, and the run resumes from exactly where it paused with that decision injected.
Problem
You run a customer-support agent for an online homeware shop. It chats with customers and has tools to look up orders, issue refunds, draft and send emails, and change shipping addresses. A customer writes: "Order 981 arrived broken, I want my $480 back." Shop policy says support staff can refund up to $250; anything above goes to finance.
Left alone, the agent reads the order, sees a damaged item and calls issue_refund(order=981, amount=480). Each tool call was valid. The agent simply lacked context a person had. Where this breaks:
- Irreversible side effects. Once the money is sent or the email is out, a better model on the next step cannot undo it. Anthropic notes that agent errors compound, and its own example is an expense agent that cancels a subscription the finance team still needed.
- Plausible but unwanted actions. Anthropic's post on Claude Code's auto mode names "overeager" behavior and honest mistakes as the main risks: the agent does something reasonable that nobody asked for.
- Nobody can be held accountable. In regulated settings someone must be able to override or stop the system. Article 14 of the EU AI Act requires high-risk systems to let overseers override outputs and halt the system with a stop button or similar procedure.
- The obvious fix fails too. Asking before every call trains people to click "yes". Anthropic reports that users accept 93% of Claude Code's manual permission prompts. A gate that is always approved gives little protection.
Solution
Add three things around the agent loop. A policy, written in code, decides which proposed actions need a human. A durable pause saves the whole run and returns control, so nothing has to stay running while a person thinks. A resume loads the run, applies the person's decision exactly once and continues. Everything else stays as it was.
- The agent proposes. The model emits
issue_refund(order=981, amount=480). Nothing runs yet. - The policy decides. Ordinary code, not the model, checks the call: order lookups are allowed, refunds above $50 need a human (the support lead up to $250, finance above that), and account deletion is denied outright. This refund is $480, so it goes to finance.
- Pause and persist. The runtime builds an approval request (what will happen, why, impact: irreversible, $480), saves the messages and the pending call under the thread id
ticket-981, and returns. The agent tells the customer the refund is waiting for approval. - Notify and wait. The request lands in the finance channel. It may sit there for hours. A timer runs; if nobody decides in time, the default is to not refund and escalate.
- The human decides. Alice from finance edits the amount to 250 and adds a comment: "policy cap, offer store credit for the rest".
- Resume. The app calls the run again with the same thread id and Alice's decision. The runtime checks that the decision matches the stored request, checks the order has not been refunded in the meantime, runs
issue_refund(order=981, amount=250)with the request id as idempotency key, and the model reads the comment and offers store credit. - Record. The request, the decision, who made it, when, and what actually ran go to the audit log.
Who decides the flow: the underlying pattern still decides the path (here a ReAct loop). Code decides where the run stops. The human decides only what happens at that stop. This is why Human-in-the-Loop is not a topology like a supervisor or a swarm: you add it to any of them.
The closest neighbour is Evaluator–Optimizer with a human as the evaluator. An evaluator scores output after it exists; an approval gate stops an action before it has any effect, which is what matters for sends, payments and deletes.
Real-world analogy
A junior clerk at a bank can look up accounts and prepare transfers alone. Any transfer above a limit goes into a supervisor's approval tray as a filled-in form. The clerk moves on to other work; the supervisor signs, corrects the amount or sends it back with a note. The clerk is the agent, the limit table is the policy, the form is the approval request, the tray is the persisted state, and the supervisor is the reviewer.
The analogy stops in two places. A clerk remembers the case while waiting; an agent remembers nothing unless its state was saved. And a supervisor who signs two hundred forms a day without reading them is the rubber-stamping problem this pattern has to design against.
Structure
- Agent loop. Any agent or workflow: a ReAct loop, a plan executor, a supervisor's worker. It proposes a tool call, a plan, a draft or a question.
- Policy gate. Code outside the model that maps each proposal to allow, ask or deny, using the tool, its arguments, a risk tier, the permission mode and any remembered "always allow" rules. A denied call goes back to the model as a tool error with a reason, so it can re-plan.
- Tool execution. Allowed calls run straight away; approved or edited calls run after resume. Every gated call carries an idempotency key so a retry or a double click cannot run it twice.
- Pause and save. Builds the approval request (summary, reason, arguments, impact, allowed decisions, expiry) and saves messages, pending calls and position to a durable store keyed by thread id. Then it returns; no process waits.
- Reviewer. A person with authority and context, reached where they already work. A timer applies a safe default (do not act, or escalate to another approver) if nobody answers.
- Resume. Authenticates the reviewer, binds the decision to the stored request id, re-checks preconditions after a long wait, applies the decision once, and puts the result (or the rejection message) into the conversation as the tool result.
- Audit log. An append-only record of each request, decision, reviewer, time and what actually ran. It also feeds the approval, edit and rejection rates you use to tune the policy.
Prompt generator
List the agent's tools and say which ones run freely, which need a human and which are forbidden. The generator writes the section of the agent's system prompt that explains gates and rejections, a gate policy to implement in your harness (LangGraph, OpenAI Agents SDK, Claude Agent SDK or your own loop), the approval request the reviewer sees, and optionally a prompt for an automated pre-reviewer.
Generated prompt
Highlighted text comes from the form. [Bracketed] text is still to fill in.
The system prompt only shapes what the model tries. Which calls pause must be enforced by the harness before any tool runs, as Claude Code's documentation stresses for its own permission rules. Treat Part 2 as a spec for that code. The quality of the approval request (Part 3) then decides whether reviewers read it or click through.
Applicability
Use it when
- Actions are irreversible or costly. Payments, external messages, deletes, deploys and permission changes cannot be fixed by a better next step.
- Intent is ambiguous or preferences matter. The agent acts for someone who may want a different tone, design or trade-off than the one it would pick.
- Errors have legal, safety or reputational impact. Or regulation requires oversight, as Article 14 of the EU AI Act does for high-risk systems.
- The agent works on systems you cannot sandbox. Production databases and real customer accounts.
- You are early in deployment. Start gated, measure approval and edit rates, and relax the tiers that are always approved.
- The agent needs something only a person has. Credentials, access or a business decision. In Anthropic's data, 12% of Claude Code's self-initiated stops were requests for credentials or access.
Avoid it when
- Actions are read-only or fully reversible in a sandbox. Let them run and review the result, or audit afterwards.
- Reviewers approve nearly everything without reading. That gate is theater. Replace it with sandboxing, an allowlist, an automated classifier or post-hoc audit.
- Latency is critical and no human can answer in time. Design safe defaults instead of waiting.
- The reviewer cannot judge the request. Approving opaque shell commands adds liability, not safety. Improve the request or route it to someone qualified.
- The risky step can be designed away. In a fixed chain, write a draft instead of sending, or make the step reversible.
How to implement
- Inventory tools and outputs. For each one: does it change state outside the sandbox, can it be undone, who is affected, what does it cost, does data leave?
- Assign risk tiers and a policy per tier. Approval should follow risk, irreversibility and uncertainty, not how important a tool sounds. Unknown or new tools default to ask (fail closed).
- Evaluate deny before ask before allow. Claude Code checks rules in that order and the first match wins, so a narrow allow cannot undo a broad deny. Ask when arguments cannot be parsed: the OpenAI Agents SDK requires manual approval when arguments are malformed, and Claude Code prompts for shell commands it cannot fully parse.
- Pick where the gate goes. Before a tool runs (approval), after an artifact is produced (review), at the plan (plan approval), or at a handoff (escalation).
- Use a durable state store and a thread id scheme. In-memory checkpointers are for tests. Do not store secrets in serialized run state.
- Define the request and decision contracts. The server holds the pending request; the decision only references it by id. Validate edited arguments against the tool schema, and escape model-written text when you render it, because the model may have read injected content.
- Build the reviewer surface and notifications. Choose sync or async, set a timeout with a safe default, and route requests to the person who owns the resource.
- Make side effects idempotent and resume atomic. Attach an idempotency key to every gated call and use a compare-and-set on run status, so two clicks or two workers cannot execute twice.
- Feed rejections back to the model. The reviewer's reason comes back as a tool error, so the agent re-plans instead of retrying the same call.
- Add stop conditions and measure. Max iterations, denial thresholds, budgets and an emergency stop. Track approval, edit, rejection and timeout rates and time to decision, then tune tiers so people see few, meaningful requests.
| Tier | Examples | Policy |
|---|---|---|
| T0 read-only | Search, read a file, look up an order | Allow and log |
| T1 reversible, local | Edit files in a git workspace, create a draft | Allow inside the sandbox |
| T2 external but bounded | Shell commands, internal APIs, fetching a new domain | Allowlist known-safe patterns; ask for the rest, or let a classifier decide with a human fallback |
| T3 consequential | Email customers, refund under the cap, deploy to staging, merge a PR | Ask a human |
| T4 critical or irreversible | Payments above the cap, production deletes, permission changes, legal commitments | Ask a qualified human, possibly two; or deny for the agent |
The pause and resume path, independent of any framework (simplified from the research notes):
def run(thread_id, decisions=None):
state = store.load(thread_id)
for d in decisions or []: # resume path
pending = state.pending.pop(d.request_id) # must match a stored request
assert authorized(d.reviewer, pending)
if d.type in ("approve", "edit"):
call = pending.call if d.type == "approve" else pending.call.with_args(validated(d.args))
state.messages.append(execute(call, key=pending.request_id))
else: # reject: the reason goes to the model
state.messages.append(tool_error(pending.call, d.message))
audit.log(pending, d)
for step in range(MAX_STEPS): # normal loop
reply = llm(state.messages, tools)
if not reply.tool_calls:
store.save(state); return Done(reply.text)
for call in reply.tool_calls:
verdict = gate(call) # code, not the model
if verdict == "allow": state.messages.append(execute(call, key=call.id))
elif verdict == "deny": state.messages.append(tool_error(call, "Blocked by policy"))
else: state.pending[new_id()] = ApprovalRequest.from_call(call)
if state.pending: # pause path
store.save(state); notify(state.pending); return Paused(state.pending)
return Escalate("max steps reached")
| Framework | How this pattern appears |
|---|---|
| LangGraph | interrupt(value) inside a node pauses the graph and saves state through the checkpointer; Command(resume=...) with the same thread_id continues. The node restarts from the top on resume, so put side effects after the interrupt. Static interrupt_before/interrupt_after are meant for debugging. |
| LangChain agents | HumanInTheLoopMiddleware with interrupt_on per tool and decisions approve, edit, reject, respond. Agent Inbox is an open-source reviewer UI for these interrupts. |
| OpenAI Agents SDK | needs_approval on a tool (a boolean or a function of the arguments); the run returns interruptions; serialize RunState, then approve or reject and run again. The SDK does not authenticate reviewers or snapshots: your server must. |
| Claude Code / Claude Agent SDK | Allow, ask and deny rules plus permission modes (default, acceptEdits, plan, auto, dontAsk, bypassPermissions), the canUseTool callback, AskUserQuestion, and a PreToolUse hook that can defer so the process exits and resumes later. |
| Microsoft Agent Framework | Approval-required function tools; the run returns approval requests instead of an answer, and responses are bound to pending requests in the same session. |
| Google ADK | FunctionTool(require_confirmation=...) or tool_context.request_confirmation(...); LongRunningFunctionTool pauses the run until the client continues. |
| CrewAI | Task(human_input=True) for review of a final answer; @human_feedback in Flows routes on the verdict, with a default_outcome. |
| Temporal | The workflow waits on a Signal with a timeout. Waiting uses no compute, timers survive crashes, and the workflow history doubles as an audit trail. |
| AutoGen | UserProxyAgent (blocking, in process) or HandoffTermination(target="user") to stop the team and resume after collecting input. |
Caveat. These APIs move fast. Version-gated details (LangGraph's response_schema on interrupt(), LangChain's when and per_call options, Microsoft Agent Framework renames) reflect the docs as read in November 2026. Google ADK's docs say tool confirmation is not supported with DatabaseSessionService or VertexAiSessionService, which limits durable async approval there. The HumanLayer approval SDK, often cited for Slack and email approvals, should be treated as historical: its repository now describes a different product. The Microsoft Agent Framework, CrewAI and Temporal details above were sourced but not re-checked, and neither were the exact semantics of the Claude Agent SDK's defer hook decision or the limits of its AskUserQuestion tool.
Variants
Tool-call approval gate
The default form: specific tools, or calls matching a condition, pause for approve, edit or reject before they run. Every major framework supports it natively. The ADK docs' example asks only when an amount exceeds 1000.
Plan approval
The agent drafts a plan, a human approves or edits it, and the steps then run without per-step prompts. One review covers many actions, so people are asked less often and each request matters more. Claude Code's plan mode explores read-only before presenting a plan. See Plan-and-Execute.
Output review (draft, then send)
The agent produces an email, PR or report and a human reviews it before release. In the running example, the agent drafts the customer email freely and only sending it is gated.
Clarifying question, or human as a tool
The model calls an "ask the human" tool when it is unsure and gets the answer as the tool result. Anthropic found that on the most complex tasks Claude Code stops to ask for clarification more than twice as often as humans interrupt it. Model-initiated questions complement a code policy; they do not replace it, since knowing when to ask is itself an open problem (Feng et al.).
Escalation or handoff to a human
The agent transfers the conversation to a person when it cannot or should not continue, as support agents do with live staff. AutoGen implements it as HandoffTermination(target="user"). See Swarm / Handoffs.
Classifier as approver, with human escalation
A model reviews routine actions and only blocked or ambiguous ones reach a person. Claude Code's auto mode works this way and hands control back to the human after 3 consecutive or 20 total denials (in headless mode it stops instead). Anthropic measured the full pipeline at a 0.4% false-positive rate on 10,000 real tool calls and a 17% false-negative rate on 52 real overeager actions, and says this is not a replacement for careful human review on high-stakes infrastructure (vendor-published).
Progressive trust and permission modes
Start restrictive and loosen as trust grows: "always allow" rules scoped to a tool and argument pattern, or a mode that auto-approves in-project edits. Anthropic reports that full auto-approve rises from about 20% of sessions for new Claude Code users to over 40% by 750 sessions.
Human on the loop
No blocking gates: a person watches a live activity stream and can interrupt, redirect or stop. In the same Anthropic data, interrupts rise from about 5% to about 9% of turns with experience, so oversight shifts from approving each step to monitoring.
Post-hoc review
Actions run, and humans review a sample or all logs afterwards. This is auditing, not gating, and only fits reversible or low-impact actions.
State editing, rewind and fork
The human edits the agent's state, or rewinds to an earlier checkpoint and forks a new branch (LangGraph update_state on an old checkpoint). Useful to correct a wrong assumption made several steps back.
Four-eyes (two approvers)
Two qualified people must confirm. The EU AI Act requires this for acting on the output of certain remote biometric identification systems. Use it for the T4 tier.
Autonomy levels
Feng, McDonald and Zhang describe five user roles: operator, collaborator, consultant, approver and observer. Most implementations on this page are consultant or approver designs; at observer level the only control left is an emergency off-switch.
Pros and cons
Pros
- Catches plausible but unwanted actions before they have an effect, which no amount of after-the-fact checking can do for a payment or a sent email.
- Every gated decision becomes an explicit, logged event with a named reviewer, which helps debugging and compliance.
- Edits are labelled data: a high edit rate tells you exactly where the agent's proposals are weak.
- Works with any pattern, and the major frameworks ship the pause and resume mechanics.
- Lets you deploy early and loosen gates based on measured approval rates instead of guesses.
Cons
- Value depends on attention. With 93% acceptance of prompts in Claude Code, much of it is lost unless requests are rare and meaningful.
- Latency goes from seconds to days, and a run is only as fast as its slowest approver.
- Human attention becomes the throughput limit; human time costs far more than tokens.
- You own durable state, idempotency, resume semantics, reviewer auth, notifications and the review UI. Frameworks only cover pause and resume.
- Rubber-stamped approvals can shift blame to people who had no realistic chance to catch the error.
Failure modes
- Approval fatigue
-
Reviewers approve without reading. Sub-second approvals on complex requests and approval rates near 100% are the signs.
Mitigation: gate fewer, higher-stakes actions and sandbox the rest (Anthropic reports sandboxing cut Claude Code's permission prompts by 84% internally). Show risk labels, track time to decision, and consider seeding known-bad canary requests to measure the catch rate, used carefully and transparently.
- Automation bias
-
People over-trust the agent's proposal or miss what it did not flag. In Goddard's study of 26 UK general practitioners, advice that was 70% accurate raised overall accuracy, but in 5.2% of cases a doctor switched from a correct answer to the system's wrong one. This evidence comes from clinical and aviation settings, not from LLM approval screens.
Mitigation: show the evidence, not only the recommendation; make reviewers accountable for their decisions; require a justification or edit for the highest tier.
- Uninformative request
-
"Allow tool run?" followed by raw JSON. The reviewer cannot judge it and clicks yes.
Mitigation: a request builder that shows a summary, the agent's reason, the impact, a diff or preview, and whether it can be undone.
- Misleading request (prompt injection)
-
The agent read injected content and writes a harmless-looking description for a harmful call.
Mitigation: render the exact arguments next to the model's summary, escape untrusted text, and keep deny rules and a sandbox that no approval can override.
- Over-broad "always allow"
-
A remembered approval for one context applies elsewhere: to another agent after a handoff, or to a replacement tool with the same name. OpenAI and Microsoft both document this risk for their sticky approvals.
Mitigation: scope rules to tool, argument pattern, agent and workspace; expire them; review saved rules when tools change.
- Forged or replayed decisions
-
A client submits an approval for a call that was never pending, or with different arguments.
Mitigation: keep pending requests on the server, bind each decision to a request id and session, authenticate the reviewer from your app's auth, and accept new arguments only through a validated edit.
- Double execution on resume
-
Two clicks, a retry, or a crash after the tool ran but before state was saved repeats the refund. The OpenAI docs warn that approval alone does not give exactly-once execution.
Mitigation: idempotency keys, atomic state transitions, and reconciling with the payment system before retrying.
- Side effects re-run before the pause
-
In LangGraph, code before
interrupt()runs again on resume and duplicates records. A baretry/exceptaroundinterrupt()swallows the pause.Mitigation: put side effects after the interrupt or in another node, use upserts, and follow the framework's ordering rules.
- Stale approval
-
Approved yesterday, executed today on a world that changed: the order was already refunded by a colleague.
Mitigation: expiry on requests, re-checked preconditions at resume, and the creation time shown to the reviewer.
- No timeout, or a timeout that approves
-
Runs hang forever, or silence turns into consent.
Mitigation: durable timers whose default is to not act, or to escalate to the next approver.
- Denial loops and gate bypass
-
The agent retries a blocked action, rephrases it (
/bin/rminstead ofrm,sh -c), or splits it into allowed steps. Claude Code's docs warn that a deny rule oncurldoes not cover the same program called by path.Mitigation: rejection messages that explain and suggest alternatives, denial counters that stop the run, and an OS-level sandbox, since rules on command text are not a security boundary.
- Checks silently skipped
-
In the Claude Agent SDK, calls auto-approved by a rule or mode never reach
canUseTool, so checks placed there do not run.Mitigation: put mandatory checks in a
PreToolUsehook, which runs for every call. - Human bottleneck
-
The queue grows, agents idle, and users switch to a bypass mode.
Mitigation: measure queue length and wait time, automate where approval is close to 100%, add reviewers, and batch related requests.
Relations with other patterns
- Prompt Chaining: a human gate between steps, such as approving the outline before drafting.
- Routing: one route is a human queue, used for low-confidence or sensitive classifications.
- Parallelization: parallel branches can each interrupt and be resumed in one batch; a human can break a voting tie.
- Evaluator–Optimizer: the human is the evaluator, or the escalation target when the loop hits its iteration cap.
- ReAct (tool-use loop): the most common insertion point. The gate sits between "model proposes a tool call" and "harness runs it", and asking the human can be one of the tools.
- Plan-and-Execute: approve or edit the plan before execution, and again when the agent replans.
- Orchestrator–Workers: gate worker tool calls or let the orchestrator escalate failed subtasks; in the OpenAI Agents SDK, approvals raised in nested runs surface at the top level.
- Supervisor: the supervisor can route to a human as if to another worker, and a human can override its choices.
- Hierarchical teams: escalation travels up the tree, with a human above the top manager for decisions beyond its authority.
- Swarm / Handoffs: a handoff target can be a person; remembered approvals must not leak from one agent to another across a handoff.
- Group Chat: the human is a participant, or takes the speaker-selector role.
- Blackboard: people post to the board or validate entries before they become authoritative.
- Council / Debate: a human acts as judge or tie-breaker over the members' positions.
Further reading
- Interrupts (LangGraph docs):
interrupt()andCommand(resume), why nodes re-run on resume, and the rules that keep pauses from breaking. - Human-in-the-loop (OpenAI Agents SDK docs): tool approval, serializing run state, and a clear section on securing approvals and avoiding double execution.
- Configure permissions (Claude Code docs): allow, ask and deny rules, their evaluation order, permission modes, and why rules on command text are not a security boundary.
- Claude Code auto mode (Anthropic): the 93% acceptance figure, a classifier that replaces most prompts, and its measured miss rates.
- Measuring AI agent autonomy in practice (Anthropic): how users' approval and interrupt behavior changes with experience, and why agents stop to ask.
- Levels of Autonomy for AI Agents (Feng, McDonald, Zhang, arXiv): five roles for the user, from operator to observer, and the open questions on rubber-stamping and when to ask.
On the numbers. The 93% figure comes from Anthropic's auto-mode post (March 2026), with no published sample size or period. Secondary coverage cites 97% from a later Anthropic announcement that could not be located, so it is not used here. Anthropic's statistics on human involvement in API traffic are self-reported upper bounds from one vendor's platform.