Multi-agent systems for complex workflows: when they pay off and how to build them

Every agent framework now ships some way to make agents talk to each other: handoffs, crews, group chats, supervisors, agent cards. The demos are compelling. The production reality is that a multi-agent system multiplies cost, latency and the number of places a run can go wrong, and the failures are harder to see because they live in the gaps between agents.

Multi-agent architectures are still the right answer for some workflows. This guide is how we decide when they are, which topologies we actually ship, how we pass context and state between agents, where MCP and A2A fit, and what we instrument so the system stays debuggable. It assumes the single-agent basics (budgets, timeouts, idempotent tools, approvals) are already in place; we cover those in our guide to reliable agents in production.

Start with the case against multiple agents

Our default is one agent with well-designed tools, wrapped in as much deterministic code as the workflow allows. Anthropic’s widely cited guidance draws the line we use: workflows orchestrate models and tools through predefined code paths, while agents let the model direct its own process, and you should find the simplest solution that works and add complexity only when it is needed.¹ Many “multi-agent systems” we review are really a fixed workflow, and are more reliable once rewritten as one.

The costs of splitting are concrete:

• Tokens. Anthropic reported that in its research product, agents used about four times the tokens of chat interactions and multi-agent systems about fifteen times.² The economics only work when the task is valuable enough to pay for that.
• Lost context. Cognition’s 2025 essay framed the core problem in two principles: share full agent traces, not just individual messages; and remember that actions carry implicit decisions, so agents acting on conflicting assumptions produce bad results.³ Its April 2026 follow-up reports that parallel agents writing at the same time still make conflicting choices about style and edge cases.⁴
• New failure classes. A Berkeley-led study of more than 1,600 annotated traces across seven multi-agent frameworks catalogued 14 failure modes in three groups: system design issues, inter-agent misalignment, and task verification.⁵ Most of these cannot happen in a single agent.

The same Anthropic post is also the clearest evidence for when splitting pays off. Its multi-agent research system, with a lead agent directing parallel subagents, outperformed a single agent by 90.2% on the company’s internal research eval, and the authors note that token usage alone explained 80% of performance variance on the BrowseComp benchmark. They are equally clear about the limits: domains where all agents need the same context, or with many dependencies between agents, are a poor fit, and they call out coding as having fewer truly parallel tasks than research.² Multiple agents mostly buy you more total reasoning spread across separate context windows. That helps on broad, parallel, read-heavy work and hurts on tightly coupled work.

Five patterns that hold up in production

When the bar is cleared, almost everything we ship reduces to one of five topologies. They map closely onto the workflow patterns in Anthropic’s guide (prompt chaining, routing, parallelisation, orchestrator–workers and evaluator–optimiser) and onto the orchestration options in the OpenAI Agents SDK documentation.¹⁶

1. Pipeline

Agents run in a fixed order and each output becomes the next input. Routing lives in code, not in a model. OpenAI’s documentation makes the trade-off explicit: orchestrating in code is more deterministic and predictable in speed, cost and performance than letting a model plan.⁶ Use it when the stages are stable and you want a trace that reads top to bottom. Avoid it when inputs vary so much that many runs need to skip or repeat stages; you will end up encoding an agent’s judgement in brittle conditionals.

2. Router and handoffs

A triage agent classifies the request and transfers control to a specialist, which owns the rest of the conversation. In the OpenAI Agents SDK, handoffs are presented to the model as tools (a handoff to a refund agent appears as transfer_to_refund_agent), the receiving agent sees the prior conversation by default, and an input_filter lets you control what it sees.⁷

from agents import Agent, Runner, handoff

billing = Agent(name="Billing", instructions=BILLING_PROMPT,
                tools=[get_invoice, explain_charge])
refunds = Agent(name="Refunds", instructions=REFUNDS_PROMPT,
                tools=[get_order, request_refund])       # write tool: gated

triage = Agent(
    name="Triage",
    instructions="Route to exactly one specialist. Never answer billing or refund questions yourself.",
    handoffs=[billing, handoff(refunds, input_filter=strip_internal_notes)],
)

result = await Runner.run(triage, user_message, max_turns=12)

Use it for distinct intents with different tools, permissions or owning teams. Avoid handoff chains deeper than one or two hops; each hop is another chance to lose context or bounce back and forth. Cap handoffs per run and log the reason for every transfer.

3. Orchestrator–workers

An orchestrator decomposes the task, calls workers with bounded sub-tasks, and synthesises the results. Unlike a handoff, control never leaves the orchestrator; in the OpenAI SDK this is the “agents as tools” style, where a manager calls specialists through Agent.as_tool() and keeps ownership of the reply.⁶ Cognition describes running this “map-reduce-and-manage” shape in production while noting that coordination still needs work: managers tend to be over-prescriptive, and children assume shared state they do not have.⁴ Use it for breadth: research, due diligence, multi-source analysis. Avoid recursion (workers that spawn orchestrators) and unbounded decomposition. We cap the number of sub-tasks per run and the depth at one.

4. Parallel fan-out and fan-in

A deterministic split sends independent slices of work to parallel agents, then a merge step combines them. This is the orchestrator pattern with the planning taken out of the model’s hands, or Anthropic’s “sectioning” and “voting” variants of parallelisation.¹ It is the cheapest way to cut wall-clock time on large inputs: one agent per contract, per region, per document. The hard part is partial failure, which we cover with code below.

5. Generator plus critic

One agent produces; another reviews against explicit criteria and sends back issues; the loop stops on a pass or after a fixed number of rounds. This is the pattern we see pay for itself most reliably. Cognition reports that a reviewer starting from a clean context catches issues the coding agent missed.⁴ The critic should get the brief and the artefact, not the generator’s reasoning, or it will inherit the same blind spots.

MAX_ROUNDS = 3

draft = await generator.run(brief)
for _ in range(MAX_ROUNDS):
    review = await critic.run(ReviewInput(brief=brief, draft=draft))   # fresh context
    if review.verdict == "pass":
        break
    draft = await generator.run(brief, feedback=review.issues)
else:
    await escalate_to_human(brief, draft, review)      # never ship an unreviewed fail

You will notice what is missing: free-form swarms and “peer” agents negotiating on a shared blackboard. We have not seen them survive production, and Cognition’s 2026 assessment is blunter, describing unstructured agent networks as mostly a distraction.⁴

The single-writer rule

If you take one design rule from this article, take this one. Cognition’s summary of what works in 2026 is that writes stay single-threaded and additional agents contribute intelligence rather than actions.⁴ In every pattern above, exactly one agent (or your code) mutates the artefact, the database or the outside world. Workers research, extract, critique and propose. The writer decides.

This one rule removes most merge conflicts, duplicated side effects and contradictory decisions. It also simplifies security: only the writer needs write credentials, and only the writer’s actions need approval gates.

Context passing and shared state

The biggest source of bugs in multi-agent systems is informal message passing. One agent emits prose, the next guesses what it means, and two weeks later a prompt change makes the receiver misread half its inputs. MAST’s inter-agent misalignment category is largely this: ignored input, withheld information, and agents drifting from the task they were given.⁵

Write a real brief for every delegation

Anthropic found that without detailed task descriptions, subagents duplicate work, leave gaps or miss information; each one needs an objective, an output format, guidance on tools and sources, and clear boundaries.² We encode that as a typed contract on both directions of every delegation, validated on both ends, with schema violations treated as hard failures.

class TaskBrief(BaseModel):
    task_id: str
    objective: str                        # one sentence, testable
    scope: list[str]                      # what to cover
    out_of_scope: list[str]               # what siblings are covering
    allowed_tools: list[str]              # least privilege per worker
    output_schema: str                    # name of the result model
    budget: WorkerBudget                  # steps, tokens, seconds

class Finding(BaseModel):
    claim: str
    source_url: str | None                # keep evidence, not just summaries
    quote: str | None
    confidence: Literal["high", "medium", "low"]

class WorkerResult(BaseModel):
    task_id: str
    status: Literal["complete", "partial", "failed"]
    findings: list[Finding]
    gaps: list[str]                       # what was NOT covered, and why

Three fields do most of the work. out_of_scope stops siblings duplicating each other. Evidence on every finding lets the synthesiser and critic check claims instead of trusting summaries, which is how invented facts stop propagating. And gaps forces a worker to say what it could not do, so the final answer can say so too.

Fan out with timeouts and explicit partial results

async def fan_out(briefs: list[TaskBrief], run, max_parallel=4) -> list[WorkerResult]:
    sem = asyncio.Semaphore(max_parallel)

    async def one(brief: TaskBrief) -> WorkerResult:
        async with sem:
            try:
                return await asyncio.wait_for(run_worker(brief, run),
                                              timeout=brief.budget.seconds)
            except Exception as e:                       # timeout, budget, schema
                run.log_worker_failure(brief.task_id, e)
                return WorkerResult(task_id=brief.task_id, status="failed",
                                    findings=[], gaps=[f"worker failed: {type(e).__name__}"])

    results = await asyncio.gather(*(one(b) for b in briefs))
    if sum(r.status == "complete" for r in results) < len(results) // 2 + 1:
        raise InsufficientCoverage(results)              # don't synthesise from scraps
    return results

The merge step receives every result, including failures, and the synthesis prompt is required to reflect gaps. Without that, a timed-out worker turns into a confident answer built on half the data.

Shared state: one owner per field

When agents share a state object (a LangGraph state, a CrewAI Flow state, a database row), give every field a single owner and make everything else read-only to that agent. Append-only lists for findings, a single writer for the decision, and a version number on the artefact so a stale write fails loudly. Persist that state at each step; multi-agent runs are long, and you want to resume from the last good checkpoint, not restart six workers.

Failure modes and how they show up in traces

MCP and A2A: two protocols, two different jobs

The two protocols people most often confuse solve different problems. The A2A project’s own documentation puts it simply: MCP is vertical, connecting one agent to its tools, APIs and data; A2A is horizontal, connecting independent agents that can reason, plan and hold multi-turn exchanges across team or organisational boundaries.⁸

A2A (Agent2Agent) treats a remote agent as an opaque system. An agent publishes an Agent Card (a JSON document describing its identity, skills, endpoint and authentication requirements); clients create Tasks, a stateful unit of work with a lifecycle; they exchange Messages; and the agent returns Artifacts as its outputs. Communication runs over HTTPS with JSON-RPC 2.0, with Server-Sent Events for streaming and webhook push notifications for long tasks.⁹ Google launched A2A in April 2025 and donated it to the Linux Foundation; version 1.0 shipped in March 2026 with version negotiation, multi-tenancy, multiple protocol bindings and signed agent cards, and in August 2026 the project joined the Agentic AI Foundation (AAIF).¹⁰

MCP (Model Context Protocol) standardises how an agent discovers and calls tools and reads resources. Anthropic donated it to the AAIF, a directed fund under the Linux Foundation, in December 2025.¹¹ The current specification, 2026-07-28, turned MCP into a stateless request/response protocol that can sit behind ordinary load balancers, and moved long-running work into a Tasks extension.¹²

Our practical rule: inside one codebase and one team, call sub-agents in-process through your framework; there is nothing to gain from a network protocol between functions you own. Use A2A at a genuine boundary, such as another department’s agent, a vendor’s agent or a customer’s, where you cannot and should not see the other side’s internals. Expose your systems through MCP when more than one agent or client needs them. And treat everything that arrives over either protocol as untrusted input, with the same allow-listing and approval rules you apply to web content.

Choosing a framework

The frameworks have converged on similar primitives, and none of them will design your contracts, budgets or failure handling for you. Pick the one your team can read and debug, that fits your deployment environment, and that integrates with the tracing you already run. A summary of what each provides, from its own documentation:

One status note, because it still catches teams out: AutoGen’s repository now tells new users to start with Microsoft Agent Framework, advises against production use, and describes the project as transitioning to archived.²¹ If you have an AutoGen prototype, plan the migration before you plan the launch.

Observability for multi-agent runs

A single-agent run produces a list of spans. A multi-agent run produces a tree, and if you cannot read the tree you cannot debug the system. The OpenTelemetry GenAI conventions give you the skeleton: an invoke_agent span per agent invocation carrying gen_ai.agent.name and, where available, gen_ai.conversation.id and token usage, with tool and model spans nested beneath.²² On top of that we add:

• One run ID propagated through every agent, tool call, queue hop and A2A request, so a run can be pulled up as a single tree.
• The full contract payloads at every agent boundary: the brief that went in and the result that came out, not a summary.
• Per-agent cost and step counts, so a runaway worker shows up on its own line instead of inside an aggregate.
• Handoff and delegation reasons as span attributes, so you can see why control moved, not just that it did.

Evaluate the system, and beat a single-agent baseline

Evaluate end to end first. The question is whether the final outcome is correct, checked against the state of the world rather than the system’s description of it, which is the framing Anthropic’s eval guide recommends.²³ Then add component evals where failures cluster: is the orchestrator’s decomposition complete, does each worker follow its brief, does the critic catch seeded defects?

Most importantly, keep a single-agent baseline in the suite with a comparable token budget. If the multi-agent design does not beat it on outcomes by enough to justify the extra cost and latency, it has not earned its complexity. We have retired more than one multi-agent system this way, and the replacements were easier to operate.

Design checklist

1. A written reason why a single agent or a code workflow is not enough, backed by an eval.
2. One of the five patterns, chosen deliberately; no free-form swarms.
3. Exactly one writer per artefact or external system; other agents propose.
4. Typed briefs and results on every delegation, validated on both ends.
5. Evidence, confidence and gaps carried through every contract.
6. Caps on sub-tasks, handoffs, critic rounds and recursion depth.
7. Per-worker timeouts and budgets, and a minimum-coverage rule before synthesis.
8. Shared state with one owner per field, versioned and checkpointed.
9. In-process calls inside your boundary; A2A only across real team or vendor boundaries; MCP for shared tools.
10. Everything from other agents and protocols treated as untrusted input.
11. One trace tree per run with OpenTelemetry GenAI spans and per-agent cost.
12. End-to-end outcome evals, component evals, and a single-agent baseline.

The system you actually have is the one you can debug, not the one on the whiteboard. If you are weighing a multi-agent design for a real workflow, our AI agent engineering team builds and runs these systems in production.

Sources

1. Anthropic Engineering, “Building effective agents,” 19 Dec 2024 — www.anthropic.com/engineering/building-effective-agents
2. Anthropic Engineering, “How we built our multi-agent research system,” 13 Jun 2025 — www.anthropic.com/engineering/multi-agent-research-system
3. Walden Yan, Cognition, “Don’t Build Multi-Agents,” 12 Jun 2025 — cognition.com/blog/dont-build-multi-agents
4. Walden Yan, Cognition, “Multi-Agents: What’s Actually Working,” 22 Apr 2026 — cognition.com/blog/multi-agents-working
5. Cemri et al., “Why Do Multi-Agent LLM Systems Fail?” arXiv:2503.13657 (v3, Oct 2025) — arxiv.org/abs/2503.13657
6. OpenAI Agents SDK documentation, “Agent orchestration” — openai.github.io/openai-agents-python/multi_agent/
7. OpenAI Agents SDK documentation, “Handoffs” — openai.github.io/openai-agents-python/handoffs/
8. A2A Protocol documentation, “A2A and MCP” — a2a-protocol.org/latest/topics/a2a-and-mcp/
9. A2A Protocol documentation, “Key concepts” — a2a-protocol.org/latest/topics/key-concepts/
10. Agentic AI Foundation, “A2A joins AAIF,” 17 Aug 2026 — aaif.io/blog/a2a-joins-aaif
11. Model Context Protocol blog, “MCP joins the Agentic AI Foundation,” 9 Dec 2025 — blog.modelcontextprotocol.io/posts/2025-12-09-mcp-joins-agentic-ai-foundation/
12. Model Context Protocol blog, “The 2026-07-28 Specification,” 28 Jul 2026 — blog.modelcontextprotocol.io/posts/2026-07-28/
13. LangChain documentation, “What’s new in LangGraph v1” — docs.langchain.com/oss/python/releases/langgraph-v1
14. LangGraph documentation, “Checkpointers” — docs.langchain.com/oss/python/langgraph/checkpointers
15. OpenAI Agents SDK documentation, overview — openai.github.io/openai-agents-python/
16. Temporal, “Production-ready agents with the OpenAI Agents SDK + Temporal,” 30 Jul 2025, updated 23 Mar 2026 (GA) — temporal.io/blog/announcing-openai-agents-sdk-integration
17. Google, Agent Development Kit (ADK) documentation — adk.dev/
18. CrewAI documentation, “Flows” — docs.crewai.com/en/concepts/flows
19. Microsoft, “Microsoft Agent Framework Version 1.0,” 3 Apr 2026 — devblogs.microsoft.com/agent-framework/microsoft-agent-framework-version-1-0/
20. AWS, Amazon Bedrock AgentCore FAQs — aws.amazon.com/bedrock/agentcore/faqs/
21. microsoft/autogen repository README (status notice) — github.com/microsoft/autogen
22. OpenTelemetry, Semantic conventions for GenAI agent and framework spans (status: Development) — github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-agent-spans.md
23. Anthropic Engineering, “Demystifying evals for AI agents,” 9 Jan 2026 — www.anthropic.com/engineering/demystifying-evals-for-ai-agents

Framework, protocol and research details were checked against each primary source in October 2026. Frameworks in this space change quickly; confirm current feature status in each project’s documentation before committing.