Hierarchical teams
Also known as: supervisor of supervisors, multi-level supervisors, tree of agents, nested orchestration, recursive delegation, hierarchical multi-agent system (HMAS), hierarchical agent teams (LangGraph), hierarchical task decomposition (Google ADK), sub-workflows (Microsoft Agent Framework), nested subagents (Claude Code)
Intent
Put managers on top of managers. A top agent splits the goal among a few team leads, and each lead splits its part among its own workers. Briefs go down, condensed reports come up, and every agent only sees its direct reports, a short tool list and the context of its own level.
Problem
A company wants an assistant that writes market-analysis reports. A typical request: "Write a six-page analysis of the European residential heat-pump market for our strategy team." The work needs web search, page scraping, financial-data lookup, a Python sandbox for charts, outlining, drafting, fact-checking, compliance review and formatting. Call it 10 specialists with about 25 tools.
A single agent with all 25 tools and one giant prompt picks the wrong tools and fills its context with raw pages before it starts writing. AWS saw more hallucinated tool parameters and wrong tool choices in its single-agent baseline (Shu et al.). So the team moves to a supervisor that routes among the 10 specialists. That helps, but new problems show up:
- Routing gets harder. The supervisor picks among 10 worker descriptions on every step, and similar descriptions cause misrouting. LangGraph's hierarchical-teams tutorial gives exactly this as its motivation: with too many workers, one supervisor becomes less effective.
- The supervisor's context fills up. Every worker's output comes back to the same conversation. After the research phase, the supervisor holds dozens of scraped pages while it tries to coordinate the drafting.
- One prompt mixes every concern. Research strategy, writing style rules and compliance policy all live in the supervisor's system prompt.
- Nothing is reusable. The research specialists exist only as entries in one big router, so nobody can build, test or reuse "research" as a unit.
Solution
Group the workers into teams and give each team its own lead. The top manager now sees three items (research team, writing team, review team) instead of ten. The research lead sees three workers and only research context. The top manager receives one condensed research report instead of forty pages, and each team can be built and tested on its own.
- The root splits the goal by team. The root manager reads the heat-pump request, splits it at the coarsest useful grain (research, then writing, then review) and decides which team to engage first.
- It writes a brief, not a transcript. The research brief states the objective, the background the team needs, constraints (European residential market only, sources with dates), the report format and a budget. The root does not pass its whole history down. LangGraph's tutorial passes only the last message to a team.
- The lead repeats the process one level down. The research lead splits its brief among its own workers: the searcher, the scraper and the data analyst. It only chooses among its direct reports and cannot address the writing team's workers.
- Workers do the work. Each worker runs its own tool loop in a clean context and writes its full output (scraped pages, a pricing table) to a shared artifact store.
- The lead checks and condenses. The research lead reads its workers' results, checks that every number has a source and a date, and returns one report: a summary, key findings, artifact IDs, open issues and a status.
- The root integrates and continues. The root reads the report, briefs the writing team with the findings and artifact IDs, then the review team with the draft. If a report says "partial" or "blocked", the root sends a follow-up brief or tells the user.
- Recursion stops at a limit set in code. An agent stops delegating when it decides to do the work, or when the depth limit takes its delegate tool away. Workers here have no delegate tool at all.
Who decides the flow
Each manager decides, but only among its own children. The developer usually fixes the shape of the tree, and code sets the hard limits.
| Decision | Made by |
|---|---|
| Tree shape: which teams exist, who reports to whom | Usually the developer (a static hierarchy). In recursive variants the agents create children at run time (ReDel, Self-Organized Agents, Claude Code nested subagents). |
| Which child acts next, and with what brief | The parent at each level, choosing only among its own children. |
| When a team is done | The team lead (LangGraph's FINISH, ADK's finish_task), bounded by code limits. |
| Maximum depth, children per node, budget | Code or configuration (CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH, ReDel's depth limit, LangGraph's recursion_limit). Never the model alone. |
The property that makes the pattern work is scoped context: a level sees its brief, its own working state and its children's reports, and nothing else unless you add a shared store on purpose. LangGraph subgraphs with their own schema, Microsoft Agent Framework sub-workflows and ADK task-mode sub-agents all enforce some version of this.
A supervisor is one manager routing among a fixed roster, and an orchestrator is one lead that invents a batch of subtasks. A hierarchy is what you get when some of their workers are supervisors or orchestrators themselves. This page uses "hierarchical" only for two or more levels of delegation. CrewAI's "hierarchical process" and Bedrock's "hierarchical collaboration model" mostly mean a single manager level, which is the Supervisor pattern on this site.
Real-world analogy
A company preparing a market study for its board. The CEO (the root manager) asks the head of market research, the head of communications and the head of legal (the team leads) for their parts. Each head assigns the work to their own staff (the workers). The CEO gets a two-page memo from each department, not the analysts' notebooks, and the analysts never talk to the CEO.
The analogy breaks in one place. People in a company share memory, hallways and informal chat, so a department head knows things nobody wrote down. An agent knows only what is in its brief. Cognition reported that its manager agents assumed their children shared state they did not have, and that children did not send discoveries back up unless there was an explicit channel for it.
Structure
- Root manager. Owns the user's goal, the split across teams, the final synthesis and the decision to stop. It should hold the least raw detail of any agent, and it is often the strongest model: Anthropic's research system runs an Opus lead over Sonnet subagents.
- Team lead. Turns a team-level brief into worker briefs, routes among its workers, checks their outputs and writes one team report. It is a worker to its parent, so it must follow the parent's report contract.
- Workers. Run tool loops and return results. They have no delegate tool, so they cannot recurse.
- Brief and report contract. Typed messages for everything that crosses a level. A brief carries objective, background, constraints, report format and budget. A report carries status, a short summary, key findings, artifact IDs and open issues.
- Shared artifact store. Files, rows or blob IDs that hold full outputs, so reports can carry references instead of content. Anthropic uses this to avoid a "game of telephone" between agents.
- Governor. Code that enforces maximum depth, children per node, steps per lead, total budget and timeouts. A tracer that records the call tree with parent IDs usually sits next to it.
Prompt generator
Describe the overall goal and one row per team. The generator writes a shared brief and report contract, a system prompt for the root manager and one system prompt per team lead. Use them as the system prompts of the nodes in a LangGraph super-graph, ADK agents wired with AgentTool, OpenAI agents exposed with as_tool(), or Claude Code subagents.
Generated prompt
Highlighted text comes from the form. [Bracketed] text is still to fill in.
Most hierarchy failures happen at the boundaries: a brief that leaves out a decision, or a summary that drops a caveat. In TalkHier's ablation, removing the "background" part of each message hurt accuracy more than removing any other field. Spend your effort on what each brief must carry and what each report must keep, and test each team on its own before wiring it under the root.
Applicability
Use it when
- One supervisor measurably breaks down. It misroutes or degrades because it has too many direct reports or tools, and the workers group naturally into cohesive teams.
- The work divides into domains or phases. Research details stay in research and only findings move up, so the internal chatter of a team never needs to reach the top.
- Teams are owned and tested separately. Different engineers own the research team and the compliance team, and each team needs its own evaluations.
- Levels need different policies. Model tier, permissions, tools or guardrails differ per team. In Bedrock, for example, each agent has its own guardrails and knowledge bases.
- The problem is recursive. Code that decomposes into functions (Self-Organized Agents) or questions that fan out into sub-questions (ReDel on FanOutQA).
Avoid it when
- One agent or one supervisor already works. Google's scaling study (Kim et al.) found coordination gains shrink or reverse once the single-agent baseline is above about 45% success. Stay with a supervisor or an orchestrator.
- The task is sequential and tightly coupled. Every multi-agent variant degraded PlanCraft by 39 to 70% in the same study, and Claude Code's agent-teams docs recommend a single session for sequential or same-file work. Use a single ReAct agent or Plan-and-Execute.
- Latency matters. Every level adds at least one more model round trip. In the AWS paper, the one domain with an extra level was about three times slower than the single agent, though the authors attribute that to task complexity and large code exchanges rather than to depth.
- The "levels" are only role names in a fixed order. MetaGPT and ChatDev are often called hierarchical, but both run a sequential role pipeline. Use Prompt Chaining.
- You cannot afford the tokens. Every manager is another model reading briefs and reports, and costs multiply with agents and turns.
How deep?
Two delegation levels (root, leads, workers) cover nearly every documented production case, and many stop at one. That is practitioner consensus and framework defaults, not a measured result: no study in the research for this page varies depth at a fixed budget. Framework defaults lean shallow. Claude Code lets subagents nest up to three layers below the main conversation, earlier Codex docs defaulted agents.max_depth to 1, ADK task-mode agents must be leaves, and Claude Code agent teams forbid nested teams.
A rough model shows why. If each summarization step keeps a fraction r of the decision-relevant detail, a fact found at a leaf reaches the root with about rd of it after d levels: with r = 0.8, two levels keep about 64% and four keep about 41%. This is an illustration from the research notes, not a measured result, but it explains why sources prefer shallow trees whose reports carry artifact references.
How to implement
- Prove you need a level. Start with one agent, then one supervisor or orchestrator. Add a level only when you can name the breakdown: misrouting among too many workers, a supervisor context that overflows, or teams that must be owned and tested separately. Google's ADK guide says plainly not to build a nested-loop system on day one.
- Draw the tree by cohesion. Group workers so most communication stays inside a team and only summaries cross team boundaries. Split by domain or phase. A split is bad when two teams keep needing each other's intermediate state. Keep spans small: Claude Code's agent-teams guide suggests starting with 3 to 5 teammates.
- Define the contract at each boundary. A typed brief going down (task ID, objective, background, constraints, report schema, budget, depth) and a typed report coming up (status, summary, key findings, artifact IDs, open issues, confidence, cost). The
statusandopen_issuesfields let a lead say "blocked" instead of inventing a result. - Build bottom-up and test each team alone. Build each team as a self-contained unit (a compiled subgraph, an
AgentTool, a sub-workflow) and evaluate it on team-level tasks before wiring it under the root. The trick that makes this work: a team has the same interface as a worker, so the tree is built by composition. - Enforce limits in code. Per team: a
FINISHdecision, a maximum number of steps per lead, and a quality check before reporting. Globally: maximum depth, maximum total agents, a cost ceiling and a deadline. Remove the delegate tool at the depth limit and reject briefs that re-delegate the parent's task unchanged. - Condense on the way up, keep a path to detail. Each lead returns a summary plus references to full artifacts, so the root can open the detail when the summary is not enough. AWS reported that passing references instead of payloads cut output tokens per communication by about 30% (Shu et al., vendor paper).
- Trace as a tree. Every span carries task ID, parent ID, depth, team, tokens, tool calls and wall time. Plant "needle" facts or caveats at the leaves and check that they survive into the root's answer: that measures telephone-game loss directly.
- Ablate the levels. Compare the hierarchy with a single agent, a flat supervisor over all workers, and the hierarchy with one level collapsed, at a comparable token budget. Anthropic attributes about 80% of performance variance in its research eval to token usage, so an unequal budget confounds the comparison.
A minimal static version, in pseudocode. A team and a worker share one interface, so a team can be a member of another team:
def make_team(name, lead_prompt, members, llm, max_steps=20):
def run(brief):
history = [system(lead_prompt), user(render(brief))]
for _ in range(max_steps):
decision = llm.structured(history, next=[m.name for m in members] + ["FINISH"])
if decision.next == "FINISH":
return llm.structured(history + [user(REPORT_PROMPT)], schema=Report)
report = lookup(members, decision.next).run(decision.child_brief)
history.append(tool_result(decision.next, compact(report)))
return Report(status="partial", summary="step limit reached")
return Agent(name=name, run=run)
research = make_team("research", RESEARCH_LEAD, [search, scrape, data], llm)
writing = make_team("writing", WRITING_LEAD, [outline, draft, charts], llm)
review = make_team("review", REVIEW_LEAD, [verify, legal], llm)
root = make_team("root", ROOT_PROMPT, [research, writing, review], strong_llm)
| Framework | How this pattern appears |
|---|---|
| LangGraph | The "Hierarchical Agent Teams" tutorial: each team is a compiled StateGraph with its own supervisor node; the top graph passes only the last message down and gets only the team's last message back. Subgraphs nest to any level, each seeing only its own state keys. |
langgraph-supervisor | A compiled supervisor team can be a member of a higher create_supervisor; output_mode controls how much team history reaches the parent. Its README now recommends building supervisors directly with tools, and the package was reported archived in September 2026. |
| Google ADK | The tree comes from sub_agents, or from AgentTool, which turns a whole sub-agent run into one function call. In ADK 2.x, task-mode agents must be leaves. |
| OpenAI Agents SDK | agent.as_tool() nests naturally. The nested agent does not inherit the parent conversation, and there is no built-in depth limit. |
| Claude Code | Subagents can spawn subagents up to three layers below the main conversation by default (CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH; 1 disables nesting). Agent teams (experimental) do not allow nested teams. |
| Microsoft Agent Framework | A workflow nests in a workflow (WorkflowExecutor in Python, AsAIAgent() in C#) to any depth, with isolated state per level, and human-input requests bubble up through the levels. |
| AutoGen 0.4 | SocietyOfMindAgent wraps an inner team as one agent and returns a model-written summary of the inner discussion. |
| CrewAI, AWS Bedrock | Process.hierarchical and Bedrock's supervisor give one manager level. Deeper CrewAI structures are built by orchestrating crews from a Flow; a Bedrock agent can be both collaborator and supervisor, with at most 10 collaborators per supervisor. |
Caveat. These APIs move fast. Claude Code's nesting rules changed (earlier docs said subagents could not spawn subagents), the Codex max_depth default of 1 comes from earlier docs and a third-party mirror, ADK 2.x is in transition and its single-parent rule is only confirmed by a search snippet and community reports, AutoGen is reportedly in maintenance mode in favour of Microsoft Agent Framework (press-sourced, unverified), and Bedrock Agents is now "Agents Classic" in maintenance mode.
Variants
Static teams of supervisors
The developer fixes the tree: a root supervisor over team supervisors over workers, each routing turn by turn. This is the LangGraph tutorial shape and the default choice when the teams are known in advance.
Agents-as-tools tree
Each lower agent is a tool of the one above, so the parent keeps control and receives a tool result. ADK documents a three-level example (report writer, research assistant, web search and summarizer). Choose it when you want the parent to stay in charge of every call.
Dynamic recursive delegation
Agents decide at run time whether to spawn children, so the tree shape varies per task (ReDel, Claude Code nested subagents). This fits open-ended problems but needs hard depth, fan-out and budget limits. With GPT-4o, ReDel scored 0.494 against 0.394 for a single agent on FanOutQA, but its error analysis also found both over- and undercommitment, described under failure modes.
Recursive code decomposition
In Self-Organized Agents, a "Mother" agent writes a function skeleton and spawns one agent per unimplemented subfunction; at the maximum depth the spawned agent is a "Child" that implements. The authors report 71.4 Pass@1 on HumanEval with GPT-3.5 at depth 2, against 66.5 for Reflexion, though the comparison is confounded (1 versus 6 unit tests per problem).
Heterogeneous nesting
A team is not one agent but another pattern wrapped to look like one: a pipeline, a parallel fan-out, an evaluator–optimizer loop or a group chat. A common shape is a supervisor at the root, orchestrator–workers inside research and a refinement loop inside writing.
Hierarchical refinement
One subtree is an evaluation team whose lead aggregates several single-metric evaluators and returns one verdict. In TalkHier's ablation on three MMLU subjects, removing the evaluation supervisor dropped accuracy from 87.21% to 81.86%, and removing the whole evaluation team to 76.15%.
Hierarchy with lateral links
The tree stays, but some peers also message each other directly, as in Claude Code agent teams. Information moves faster, at a high price: Google's scaling study measured 515% overhead and 6.2 times the turns of a single agent for its "hybrid" architecture.
Root with a routing bypass
The root sends simple requests straight to one collaborator and uses the full hierarchy only for complex ones, as in Bedrock's "supervisor with routing" mode. It cuts latency for the easy cases.
Pros and cons
Pros
- Every agent has a smaller, focused context; the root sees only summaries.
- Each routing decision has fewer options (three teams instead of ten workers).
- Teams are units you can build, test, own and reuse separately.
- Each level can use its own model tier, tools, permissions and guardrails.
- A lead that verifies its team stops errors before they reach the top: Google's study measured 4.4 times error amplification with a coordinator against 17.2 times without one.
Cons
- Each level adds at least one model round trip per hop; parallelism only helps within a level.
- Each manager is another model reading briefs and reports, and turns grow faster than the number of agents.
- Every summarization step can drop or distort detail (the telephone game).
- A lead that does not verify passes its workers' mistakes up with extra authority.
- Failures hide inside subtrees, so you need tree-shaped traces to debug.
Failure modes
- Telephone-game loss
-
The root acts on a summary of a summary. A caveat found at a leaf ("the data only covers 2023") disappears on the way up.
Mitigation: reports carry artifact references and open issues, the root can open the raw artifacts, and report schemas are constrained. Forward a worker's message verbatim where possible (
langgraph-supervisorhas a forward-message tool for this). - Missing context going down
-
A child redoes work or contradicts a decision made above, because the brief left it out. Cognition saw manager agents assume their children shared state they did not have.
Mitigation: every brief has a background field with the decisions and constraints the child needs. Tell every agent that its children cannot see its conversation.
- Over-delegation and recursion explosion
-
Too many children for a simple task (Anthropic's early agents spawned 50 subagents for simple queries), or long chains where each agent re-delegates (ReDel's undercommitment, 44.8% of WebArena runs with GPT-4o).
Mitigation: effort rules in the prompt ("delegate only if the task has two or more parts that need real tool work"), hard depth, fan-out and budget limits in code, no delegate tool at the limit, and a check that rejects re-delegating the parent's task unchanged. Tell each agent which tools its children have, so it does not delegate work they cannot do.
- Overcommitment
-
A lead does the work itself, fills its context and forgets the task: 22.7% of FanOutQA and 41.1% of TravelPlanner runs with GPT-4o in ReDel. CrewAI managers and Claude agent-team leads are reported to start implementing instead of waiting.
Mitigation: give leads few or no work tools, tell them to wait for their workers, and monitor the size of the delegation graph.
- Misrouting across teams
-
A request goes to the wrong team because team descriptions overlap.
Mitigation: distinct, non-overlapping team cards (AWS gives the same advice for collaborators), routing evals at each level, and a "no team fits" option that returns to the caller.
- Stopping too early, or never
-
A lead declares the work finished too early, or repeats steps without noticing it is done. The MAST study of failed multi-agent traces lists premature termination, step repetition and unawareness of termination conditions among its failure modes.
Mitigation: a step limit per lead, explicit done criteria in the brief, and root-level checks of team reports.
- Unverified errors gaining authority
-
A worker's mistake passes through a lead that rubber-stamps it, and the root trusts it because it comes from "the team".
Mitigation: leads verify before reporting, reports carry status and confidence, and the root spot-checks artifacts.
- Silent subtree failure
-
An error inside a team turns into an empty or partial report that the parent treats as success.
Mitigation: a typed status field, error events tagged with the team's ID (Microsoft Agent Framework does this for sub-workflows), and a rule to fail loudly upward.
- Cross-team conflicts and orphan agents
-
Two teams write the same file or make incompatible choices; a background grandchild keeps running after its parent has finished.
Mitigation: one owner per artifact, children tracked by parent ID, timeouts, and collecting or killing children when their parent exits.
- Injected instructions relayed up
-
A web page read by a worker contains instructions, and the lead passes them up as "findings".
Mitigation: treat child output as data, scan reports before the parent reads them (Claude Code scans subagent output), and never let a relayed message grant permissions.
Relations with other patterns
- Supervisor: one manager routing among a fixed roster. Hierarchical teams are supervisors of supervisors.
- Orchestrator–Workers: one level of run-time decomposition. A hierarchy appears when some workers are orchestrators themselves.
- Routing: the root often acts as a router to teams, and a routing bypass skips the hierarchy for simple requests.
- Parallelization: the usual way a lead runs its workers, and how independent teams run side by side at one level.
- Evaluator–Optimizer: often placed as a subtree (TalkHier's evaluation team) or as a lead's quality check before it reports up.
- ReAct (tool-use loop): what each worker, and often each lead, runs inside.
- Plan-and-Execute: one agent plans and executes in one context; a hierarchy spreads the plan across agents with separate contexts.
- Prompt Chaining: a pipeline can be one team inside a hierarchy. "Company" systems such as MetaGPT and ChatDev are mostly chains with role names.
- Swarm / Handoffs: peers transfer control with no fixed tree and no return to a parent.
- Group Chat: can be wrapped as one team (AutoGen's
SocietyOfMindAgent); lateral links in a hybrid hierarchy look like a group chat. - Blackboard: the shared artifact store across teams is a light blackboard.
- Council / Debate: can sit under a lead as a jury team whose verdict is reported up.
- Human-in-the-Loop: approval requests have to bubble up through the levels to reach a person.
Further reading
- Hierarchical Agent Teams (LangGraph, archived notebook): the reference static implementation, with team supervisors and last-message passing between levels.
- How we built our multi-agent research system (Anthropic): a production lead-and-subagents system, its token costs, and why outputs go to external storage.
- Subagents (Claude Code docs): nested subagents, the spawn-depth setting and how the depth limit removes the delegate tool.
- ReDel (Zhu, Dugan, Callison-Burch, EMNLP 2024 demo): recursive delegation, its over- and undercommitment failures, and replay tooling for delegation trees.
- TalkHier (Wang et al.): nested teams with a structured message protocol and an evaluation team, with ablations.
- Towards a Science of Scaling Agent Systems (Kim et al.): when coordination helps or hurts, error amplification and overhead by architecture.