Human in the loop
With this setup, a person checks an AI system's work at chosen points and can approve it, change it, reject it or step in.
Keeping a human in the loop is a design choice. At points the builder picks, a person checks what an AI system is about to do, or has just done. That person can accept it, change it, refuse it or take over. In agent frameworks such as LangChain and the OpenAI Agents SDK, it looks like an approval gate: a stop before certain tool calls run. A tool call is the AI asking to use one of its tools.
An approval gate needs two parts: a way to stop partway through a run, and a saved record of where it stopped. In LangChain, the check is done by middleware, a layer of code that sits between the model’s answer and the tools. If a call needs review, the run halts and a checkpointer, a store that keeps a snapshot of the run, holds on to it. The app files that snapshot under a thread ID, a label it sends back later so the right run carries on. The person’s answer is then passed back to the paused run, which acts on it. In the OpenAI Agents SDK, the developer sets needs_approval on a tool. When a run reaches that tool, it stops. It hands back the waiting call, with the tool name and its arguments (the inputs to the tool), so someone can decide. Claude Code, a coding agent, asks before running most shell commands in its manual mode, but reads files in the working folder without asking.
Where a gate sits decides how much it helps. Builders usually gate steps that change things in the world, such as deleting records or moving money, and let reading and searching run freely. A gate can also be finer than a plain list of tools. LangChain lets a builder add a check on a call’s arguments, so one tool pauses for some inputs and runs straight through for others. Claude Code sorts actions into three kinds of rules. Allow rules let a tool run without asking, ask rules always prompt, and deny rules block it. When rules clash, deny beats ask, and ask beats allow.
The same idea appears in standards and law, not only in agent code. NIST’s AI Risk Management Framework treats human involvement as a sliding scale, from people doing everything to the system acting alone. It notes that some jobs need no human check at all, like making video compression better. The EU AI Act sets rules for AI systems it calls high-risk. The people watching over them must be able to ignore or reverse an output. They must also be able to bring the system to a safe stop, for example with a stop button.
An agent with tools does real work, such as writing files or running database queries, so its mistakes land in real systems.
Follow one database clean-up request through an approval gate.
- 1 · proposeTold to clear out stale database rows, the model proposes an execute_sql call that deletes rows older than 30 days.
- 2 · checkThe human-in-the-loop middleware, a layer of code between the model's answer and the tools, compares the call with a policy that lists which tools need review.
- 3 · pauseBecause execute_sql is on that list, the run stops before the query executes, and a checkpointer, a store that keeps a snapshot of the run, saves where it stopped.
- 4 · decideA reviewer sees the query and may approve or reject it, because this policy allows no edits.
- 5 · resumeApproval picks the run up again using its thread ID, the label that finds the saved snapshot, and runs the query unchanged, while rejection skips the tool and tells the model why.
The model only proposes the action; the paused run waits indefinitely until a person replies.
| Who | What they ask | What it works with |
|---|---|---|
| Support team | “Cancel this customer's order.” | A cancel_order tool that always needs approval |
| Data team | “Delete records older than 30 days.” | An execute_sql tool a reviewer can approve or reject |
| Developer with a coding agent | “Run the tests and fix what fails.” | Shell commands the agent asks before running |
| Marketing lead | “Send the launch email to the list.” | A send_email call whose recipient can be edited first |
- Listed tool calls wait for a person, so they do not run until someone replies.
- Reviewers can fix a call instead of only blocking it, for example changing an email's recipient before it sends.
- Safe calls can skip review, because a policy can auto-approve read-only tools and pause only risky ones.
- A paused run can be stored in a database and resumed later, so a reviewer need not answer at once.
- A reviewer who trusts the system too readily can wave mistakes through, a risk the EU AI Act calls automation bias.
- How far people are empowered and motivated to challenge AI output still needs study, according to NIST.
- Resuming reruns the paused step from its start, so an update made before the pause can be repeated or duplicated.
- Large edits to a proposed call can make the model rethink, repeat the tool or act unexpectedly.
Sources used
This explainer is written in original language. The links below support its factual claims.
- docsHuman-in-the-loop, LangChain · read 28 Sept 2026
- docsInterrupts, LangChain · read 28 Sept 2026
- docsHuman-in-the-loop - OpenAI Agents SDK, OpenAI · read 28 Sept 2026
- officialBuilding effective agents, Anthropic · read 28 Sept 2026
- docsConfigure permissions, Anthropic · read 28 Sept 2026
- officialArtificial Intelligence Risk Management Framework (AI RMF 1.0), National Institute of Standards and Technology · read 28 Sept 2026
- officialArticle 14: Human oversight, European Commission, AI Act Service Desk · read 28 Sept 2026