Supervisor
Also known as: manager agent, supervisor–worker, supervisor–collaborator (AWS Bedrock), agents as tools or manager pattern (OpenAI), subagents (LangChain), Crew manager (CrewAI), Magentic manager (Microsoft)
Intent
One central LLM agent owns the conversation and, step after step, picks which specialist from a fixed roster acts next, or decides to answer or stop. Each worker does its part and control always comes back to the supervisor, which then decides again.
Problem
You are building the support assistant for an online electronics store. It covers orders and shipping, refunds, the product catalog and store policies. Each domain has its own API tools, often 10 to 20 of them, its own rules, and sometimes its own owning team. A customer writes: "My last order arrived broken. Refund it to my card and tell me whether the newer model is in stock."
That message needs the orders tools, then the refund tools, then the catalog, with the order number carried from one step to the next. The obvious designs all break:
- One agent with every tool. With two domains it works. With six, the prompt carries over 100 tool schemas and several policy documents, and the model picks wrong tools and mixes up policies. LangChain measured this on a modified τ-bench retail set: the single agent's score dropped sharply once two or more extra tool domains were added, and its token use grew with each domain.
- A one-shot router. A router classifies the message as "orders" and sends it to one specialist. Nobody then issues the refund, checks the stock or verifies that each step worked, because a router never gets control back.
- Peer handoffs. If the orders agent hands off to refunds, which hands off to catalog, every agent must know every other one, no single agent owns the final answer, and guardrails must be copied into each of them.
Solution
Put one agent between the user and a small team of specialists. The supervisor holds the roster and the rules but no business tools. On every hop it reads the conversation plus the results gathered so far and makes exactly one decision: call a worker with an instruction, reply to the user, or finish. The worker runs its own tool loop and returns its result, and the supervisor decides again.
- Receive. The customer's message goes to the supervisor and only to it. This single entry point is also where input guardrails sit.
- Decide. The supervisor sees its roster (
orders_agent,refunds_agent,catalog_agent,faq_agent) and callsorders_agent: "Find customer C-77's most recent order and whether it is eligible for a damage refund." - Work. The orders agent runs its own ReAct loop with the order tools and returns
{status: done, order_id: 8812, model: X200, eligible: true}. - Return and decide again. Control comes back to the supervisor. The refund and the stock check do not depend on each other, so it calls
refunds_agent("Refund order 8812 in full to the original payment method, reason: arrived damaged") andcatalog_agent("Is the successor of model X200 in stock? Price?") in the same hop. - Check. Both results come back: refund R-551 issued, 3 to 5 days; model X300 in stock at $249. The supervisor checks that every part of the request is covered.
- Respond. It writes one reply to the customer, or forwards a worker's customer-ready text unchanged. On the next message the loop starts again at the supervisor, which still holds the whole conversation.
That turn took three supervisor calls and three worker runs. A single agent might use fewer calls, but with every tool of every domain in its context.
The developer decides the roster, each worker's tools and prompt, what a worker sees and what comes back, and the hard limits. The supervisor LLM decides, on every hop, which worker acts next, with what instruction, and when the turn is over. Each worker decides how to do its own job.
The closest neighbour is Orchestrator–Workers, and the two words are often used interchangeably. On this site, an orchestrator invents a batch of subtasks at run time and merges the results; a supervisor routes each next step to one named specialist from a fixed roster and gets control back after every worker. This split is a convention of these notes, not an industry standard. In Swarm / Handoffs there is no center: the active agent hands control to a peer, which keeps the user.
Real-world analogy
A hospital ward's attending physician. The patient talks to the attending, who sends them to radiology, reads the scan, then asks the lab for blood work, then decides on treatment and explains it to the patient. The specialists never talk to the patient directly and never route the patient between themselves. Every result comes back to the attending, who decides the next step.
The analogy stops where the paraphrasing starts. A good physician reads the radiologist's report in full; an LLM supervisor tends to summarise worker results in its own words and can lose or change details on the way, which is the main weakness of the pattern.
Structure
- User. Talks only to the supervisor, on every turn. One user turn can contain several hops.
- Supervisor. An LLM in a loop with the roster and rules in its system prompt. It writes each worker's instruction, checks each result, asks the user when something is missing, and decides when the turn is done. It usually holds no domain tools.
- Delegation. How a decision becomes a worker run: a structured
nextfield read by a graph edge, a handoff tool, or (most common now) a plain tool whose body runs the worker. The developer controls what the worker receives: only the instruction, or part or all of the history. - Workers. A fixed, named roster. Each has a description the supervisor routes on, its own prompt and tools, and usually its own ReAct loop.
- Result return. Control goes back to the supervisor no matter what. A result adapter decides what the supervisor sees: the last message, a structured object, the full transcript, or a reference to a stored artifact.
- Rules and limits. Max hops per turn, token or cost budget, timeouts, duplicate-call blocking, and guardrails on input and on sensitive worker actions.
Prompt generator
Describe your team and its workers. The output is a system prompt for the supervisor agent, plus an optional block to add to every worker's prompt so results come back in a shape the supervisor can check. Use it with any framework where workers are exposed as tools (LangChain create_agent, OpenAI as_tool(), Strands, ADK AgentTool) or in a hand-built loop.
Generated prompt
Highlighted text comes from the form. [Bracketed] text is still to fill in.
The supervisor picks workers from their names and descriptions alone. LangChain and Google both stress this, and AWS asks you to minimise overlapping responsibilities. Say what each worker cannot do as clearly as what it can, and test routing on real messages before tuning anything else.
Applicability
Use it when
- Several domains would overload one agent. Many tools or conflicting policies make a single agent pick wrong tools; each worker sees only its own.
- Requests often need several specialists in sequence. The supervisor carries IDs and facts from one worker to the next and checks each result before moving on.
- You want one voice to the user. One place for guardrails, tone, compliance and the final answer. This is OpenAI's stated reason to choose agents as tools over handoffs.
- Different teams own different specialists. Workers can be built, tested and replaced separately without touching the others.
- You need a central audit trail. Every routing decision is an explicit step you can log.
Avoid it when
- One agent with a moderate toolset works. Microsoft, OpenAI and LangChain all say to start there. Use a ReAct loop.
- Each request belongs to exactly one domain. Use Routing, or a supervisor with a routing fast path.
- The specialist should own the conversation once engaged. Long, focused dialogues such as technical support fit Swarm / Handoffs better, with no relay layer.
- The work is a broad fan-out of subtasks invented at run time. Research or audits that split and merge fit Orchestrator–Workers.
- The steps are known in advance. Use Prompt Chaining or plain code.
- Latency budgets are tight, or the task is tightly sequential planning. AWS saw turn latency roughly double to triple, and a Google Research study found every multi-agent design lost 39 to 70% on sequential planning tasks.
How to implement
- Prove you need it. Start with one agent and tools. Move to a supervisor when you see tool overload, policy mix-ups across domains, separate owning teams, or security boundaries that require separate agents.
- Write an agent card per worker. Name, one or two sentences on what it can and cannot do, its tools, its input contract and its output contract. Keep the roster small: Bedrock caps a supervisor at 10 collaborators, and LangChain suggests listing agents in the prompt only for fewer than about 10; beyond that use a discovery tool or sub-supervisors.
- Write the supervisor prompt. Roster, how to work (one call at a time unless independent, self-contained instructions, check each result, never repeat the same call), when to answer directly, when to forward, and the hop limit with what to say when it is reached. The generator above produces this.
- Choose the delegation mechanism. Most frameworks now default to agents as tools: each worker is a tool, the supervisor is an ordinary tool-calling loop, "respond" is a message with no tool call, and the tool result is the return. The older graph style uses a structured
nextfield (worker names plusFINISH) and fixed edges from every worker back to the supervisor. - Decide what workers see and what comes back. This is the decision that matters most. Isolated input (only the instruction) keeps workers focused and cheap but needs complete instructions. Returning only the last message keeps the supervisor's context small. Ask workers for a structured result with a
statusfield (done, partial, failed, needs_user_input), the IDs found, and optional customer-ready text. - Add stop logic. Max hops per user turn, max steps per worker, token budget, timeout, and blocking of repeated identical calls. When a limit is hit, have the supervisor tell the user what is done and what is not, and escalate to a human rather than loop.
- Trace and evaluate in layers. One span per supervisor decision with a nested span per worker run. Test workers on their own datasets, then routing (expected next worker for a given state), then trajectories, then end-to-end goal success with simulated users.
| Framework | How this pattern appears |
|---|---|
| LangChain / LangGraph | Current advice is the subagents pattern: create_agent workers wrapped in @tool functions, called by a create_agent supervisor with a checkpointer. The older langgraph-supervisor library (create_supervisor) offers output_mode, add_handoff_messages and a forward-message tool. |
| OpenAI Agents SDK | Agents as tools (the "manager pattern") with agent.as_tool(...). The nested agent does not see the parent conversation unless you pass the same session. Handoffs are the contrasting mechanism. |
| AWS | Strands Agents: pass agents as tools, with a clean context per call by default. Bedrock Agents: SUPERVISOR or SUPERVISOR_ROUTER collaboration mode, at most 10 collaborators. |
| Google ADK | AgentTool(agent=...) keeps the parent in control and returns the result. transfer_to_agent hands control over instead, which is closer to routing or handoffs. |
| Microsoft Agent Framework | Magentic orchestration (MagenticBuilder): a manager picks the next agent every round from a progress ledger, with max_round_count and max_stall_count. |
| CrewAI | Process.hierarchical with a manager_llm or manager_agent that assigns the developer's tasks to crew members. |
| smolagents | managed_agents=[...] on a manager agent; each managed agent needs a name and description. |
Caveat. These APIs move fast. The langgraph-supervisor repository was archived on 2026-09-20 and its migration guide calls it no longer actively maintained. AWS documents Bedrock Agents as "Agents Classic", closed to new sign-ups and in maintenance mode, and points new builds to Strands and AgentCore. CrewAI manager problems are known only from community forum reports.
Variants
Router-node supervisor
The supervisor returns a structured next field constrained to the worker names plus FINISH, and graph edges bring every worker back. Workers usually share the full message history. This is the original LangGraph tutorial shape.
Tool-calling supervisor (agents as tools)
Workers are tools and the tool result is the return. Workers get an isolated context by default and parallel calls come for free. A single task(agent_name, description) dispatch tool can replace one tool per worker when the roster is large or changes.
Supervisor with forwarding
The supervisor, or the runtime, sends a worker's reply to the user verbatim instead of rewriting it. This avoids the lossy relay and saves tokens. LangChain's forward_message tool, Strands' direct-return delegation and Bedrock's SUPERVISOR_ROUTER mode do this.
Routing fast path
A cheap classifier sends simple requests straight to one worker and falls back to full supervision when it is unsure or the request is complex. In AWS's study a Claude 3 Haiku classifier reached 0.90 to 0.92 accuracy and added about 630 to 750 ms per turn.
Parallel-dispatch supervisor
The supervisor may call several workers in one hop, as in the refund and stock check above. The more it does this, the closer it gets to Orchestrator–Workers.
Ledger-based manager
The supervisor keeps an explicit plan and a per-round progress ledger (is the request satisfied, are we looping, is progress being made) and replans on stalls. This is Microsoft's Magentic-One design.
Other variants
- Task-list manager. The developer defines the tasks and the manager assigns each to a crew member and validates the output (CrewAI).
- Async supervisor. Worker calls start background jobs and the supervisor keeps talking to the user meanwhile (LangChain subagents, async mode).
- Learned supervisor. The routing policy is trained with reinforcement learning instead of prompted (Dang et al., "puppeteer"). Results are reported at abstract level only and were not checked for this page.
- Multi-level. Workers are themselves supervisors; see Hierarchical teams.
Pros and cons
Pros
- Scales with the number of domains. In LangChain's τ-bench test the supervisor's score and token use stayed roughly flat as distractor domains were added, while the single agent fell off.
- Higher success on multi-domain, tool-heavy tasks. AWS's study (vendor research, 90 scenarios) reported 0.90 goal success for a supervisor team against 0.53 to 0.80 for a single agent.
- One entry point for guardrails, approvals, tone and logging.
- Workers stay small and testable in isolation, and different teams can own them.
- Central checking filters worker errors: Google Research measured error amplification of 4.4x for centralized designs against 17.2x for independent agents.
- A strong supervisor can drive cheaper workers: AWS reached 0.80 to 0.87 with Haiku 3.5 specialists under a Sonnet supervisor.
Cons
- An extra LLM call per hop. LangChain's docs walk through a one-domain request costing 4 calls with subagents against 3 with handoffs (an illustrative example, not a benchmark).
- More latency. AWS measured 24 to 169 seconds per turn against 9 to 53 seconds for a single agent.
- The relay loses detail. In LangChain's benchmark swarm scored slightly better than supervisor in every setting, mostly because the supervisor rephrased worker answers.
- Single point of failure: a wrong early routing decision or a thin instruction poisons the later hops.
- Stateless workers redo work on repeat requests unless you persist their state.
Failure modes
- Telephone game (lossy relay)
-
The supervisor paraphrases worker answers and drops details, changes numbers, or regenerates large outputs badly. It was the main cause of the supervisor's gap in LangChain's benchmark.
Mitigation: forward customer-ready replies verbatim, return structured results with IDs, and pass large artifacts by reference. LangChain's fixes (strip handoff messages from worker context, add a forward tool) gave their supervisor a "nearly 50%" improvement over their first version. AWS's payload referencing raised goal success on code tasks from 0.73 to 0.90.
- Worker lacks context
-
An isolated worker receives "refund the order" without the order ID or constraints, and fails or guesses.
Mitigation: a prompt rule that every instruction is self-contained, typed input schemas on the worker tools, or passing selected state keys.
- Mis-routing
-
Vague or overlapping descriptions send requests to the wrong specialist. You see it as a high rate of failed or "not my job" worker results.
Mitigation: precise, non-overlapping worker cards, enum-constrained routing output, and routing evals that track confusion between pairs of workers.
- Ping-pong and loops
-
The supervisor calls the same worker again and again, or alternates between two. The MAST failure taxonomy counts step repetition at 15.7% of observed multi-agent failures.
Mitigation: a hop cap, blocking of repeated identical calls, stall counters with replanning (Magentic's
max_stall_count), and escalation to a human. - Never finishes, or finishes early
-
It keeps going after the job is done, or declares done with parts missing. MAST counts 12.4% of failures as unawareness of termination conditions and 6.2% as premature termination.
Mitigation: an explicit completion checklist in the prompt, a
statusfield on every worker result, and a final verification step. - Supervisor does the work itself, or over-asks
-
It answers from its own knowledge instead of delegating, or keeps asking the user questions a worker could answer. AWS observed the second one.
Mitigation: no domain tools on the supervisor, a rule against asking for what a worker can look up, and monitoring of the delegation and clarification rates.
- Supervisor context bloat
-
Full worker transcripts and routing messages pile up in the supervisor's window over a long session, so cost rises and attention degrades.
Mitigation: return only the last message or a structured result, strip routing messages, and trim or summarise the supervisor's history.
- Prompt injection through workers
-
Untrusted content fetched by a worker reaches the supervisor as a "result" and steers its next decision.
Mitigation: treat worker output as data, give each worker the fewest tools it needs, and require approval for sensitive actions.
- Parallel calls conflict
-
Two workers called in the same hop act on the same resource.
Mitigation: allow parallel calls only for read-only or independent workers, and keep writes serial.
Relations with other patterns
- Routing: a one-shot classifier that never gets control back. A supervisor is a router that runs every hop, keeps the conversation and checks results. AWS's routing fast path combines the two.
- Orchestrator–Workers: same hub-and-spoke shape and often the same agents-as-tools code, but the orchestrator invents a batch of subtasks and merges them, while the supervisor routes each next step to a fixed roster. Many sources use the two words interchangeably.
- ReAct (tool-use loop): a tool-calling supervisor is a ReAct agent whose tools are agents, and each worker is usually a ReAct agent too.
- Plan-and-Execute: combines well. A supervisor may keep a plan (Magentic's ledgers), and a worker may plan and execute internally, as Fastweb's use-case agents do.
- Evaluator–Optimizer: a supervisor can send a draft to a reviewer worker and back, which gives a maker–checker loop driven by the supervisor.
- Hierarchical teams: supervisors whose workers are supervisors. The next step when one roster grows too large.
- Swarm / Handoffs: the active agent hands off directly to a peer, keeps control and answers the user itself. No center, no relay.
- Group Chat: a selector picks the next speaker in a shared transcript. Microsoft describes Magentic as the group-chat architecture with a stronger manager.
- Blackboard: workers coordinate through a shared store instead of through the supervisor. Shared state keys in a supervisor graph are a mild version.
- Council / Debate: several agents answer the same question; a supervisor gives each worker a different job.
- Human-in-the-Loop: approval before sensitive worker actions, and escalation when the supervisor hits its limits.
Further reading
- Benchmarking multi-agent architectures (LangChain, 2025): supervisor against swarm and a single agent on τ-bench with distractor domains, the telephone-game explanation and the fixes. Scores are only in charts, and multi-hop tasks were not tested.
- Towards Effective GenAI Multi-Agent Collaboration (Shu et al., AWS): the supervisor design behind Bedrock multi-agent, with goal-success, latency, payload-referencing and routing numbers. Vendor research on 90 scenarios.
- Subagents (LangChain docs): the current tool-based supervisor, isolated versus forked inputs, and one tool per agent versus a single dispatch tool.
- Multi-agent orchestration (OpenAI Agents SDK): agents as tools versus handoffs, and when to choose each.
- AI agent orchestration patterns (Microsoft Azure Architecture Center): why a single agent is often the right default, and reliability advice for multi-agent designs.
- Why Do Multi-Agent LLM Systems Fail? (Cemri et al.): the MAST failure taxonomy, including step repetition and termination failures.