01pipeline · prompt chaining
Sequential
Agents in a fixed order, each working on the last one’s output. Best for staged refinement; an early error carries forward.
Guide · Agent mesh
Multi-agent system architecture: how agents divide work, talk and stay correct.
A multi-agent system is several AI agents with separate roles, tools and context, plus the coordination, communication and runtime that make them one system.
Designing one is mostly about the parts between the agents: who decides which agent works next, how results and state move, which protocol carries them, and what checks the whole. This guide lays out the layers, the five common orchestration patterns, the protocols, a worked design and the points where these systems fail.
Updated · 11 sources
Multi-agent systemspecialised agents, coordinated as one
Agentseach with a role, instructions, a model and tools
Coordinatoran orchestrator, a manager agent or fixed code
Messagestool results, hand-offs, a shared thread or events
Shared statewhat every agent may read, and who may write it
Tools and datareached over APIs or MCP
Verificationa checker agent, tests or a person
Runtimedurable delivery, retries, scaling
Recordsevery message and step, traceable
What is multi-agent system architecture?
Multi-agent system architecture is the design of a system in which several AI agents — each with its own role, instructions, tools and context — work together on one job. It defines how work is divided (the orchestration pattern), how agents exchange results and state (messages, hand-offs, shared memory), which protocols connect them to tools and to each other, and how the whole run is verified, recovered and recorded.
Reference architecture
The six layers of a multi-agent system architecture.
Every design has these layers, whether they are drawn or not. Most failures trace to a layer nobody owned — usually communication, verification or the runtime underneath.
01
Triggers and peoplea chat, an API call, a new document, a schedule — and the people who approve
entry02
Coordinationdecides which agent runs next: fixed code, a manager agent or hand-offs
orchestration03
Agentsspecialists with a role, instructions, a model and a tool set
roles04
Communication and statehow results, context and decisions pass between agents
messagesmemory05
Tools and datasystems the agents read and change, reached over APIs or MCP
MCPAPIs06
Runtime and recordsdurable delivery, recovery, scaling, and a trace of every step
durabilityauditOrchestration patterns
Five orchestration patterns, plus the maker-checker loop: what each is good for and where it breaks.
Microsoft’s architecture guidance names five patterns, each known by several other names. Real systems combine them: a fixed sequence for intake, then concurrent analysis, then a maker-checker loop on the result.
02fan-out/fan-in · map-reduce
Concurrent
Agents work on the same input at once; results are voted, merged or summarised. Fast; contradictions need resolving.
03roundtable · debate
Group chat
Agents share one thread; a manager picks who speaks. Good for review and consensus; hard to steer with many agents.
04evaluator-optimizer
Maker-checker
A form of group chat: one agent drafts, another checks against criteria, until it passes or reaches its iteration cap and goes to a person.
05routing · triage
Hand-off
One agent at a time; each decides to answer or pass control to a better-placed agent. Watch for hand-off loops.
06dynamic · task ledger
Magentic
A manager builds and revises a plan as it goes, assigning tasks to agents that act in outside systems. Slow to converge.
Centralised to decentralised
Where control sits: from one orchestrator to a shared message bus.
OpenAI’s guide models a multi-agent system as a graph: in the manager pattern the edges are tool calls; in the decentralised pattern they are hand-offs that transfer the conversation. Further right, agents stop calling each other and publish to a bus.
Central controlDistributed control
12345
- Fixed pipelineCode runs agents in sequence, in parallel or in a loop — no model decides the order.
- Manager, agents as toolsOne agent holds the conversation and calls specialists like functions.
- HierarchyManagers of managers: a lead delegates to team leads, each with workers.
- Peer hand-offsControl passes from agent to agent; whoever holds it acts.
- Message busAgents and services publish messages to a shared, durable bus instead of calling each other directly.
Keep a checker in the middle. In controlled tests across 260 configurations, architectures without centralised verification propagated errors more than those with centralised coordination. Distribute the work; keep the verification in one place.
How agents connect
How agents talk: MCP for tools, A2A for other agents.
Inside one system, agents can share memory and call each other directly. Across systems and vendors, two open protocols divide the work. Google’s launch post put it in one line: A2A “complements” MCP.
MCP
The Model Context Protocol. An agent discovers and calls tools, reads resources and uses prompts from a server.
- Introduced by Anthropic, November 2024
- Server lists tools; model chooses one
- JSON-RPC 2.0 sessions
- OAuth for remote servers
A2A
The Agent2Agent protocol. A client agent finds a remote agent by its Agent Card and sends it tasks.
- Launched by Google, April 2025
- Agent Card advertises capabilities
- Tasks with a lifecycle and artifacts
- Linux Foundation project since June 2025
An agent to tools and dataConnectsAn agent to another agent
A capability to useThe other side isA peer with its own reasoning
A tool callUnit of workA task
Reach a CRM, a database, a file storeTypical useDelegate to another team’s or vendor’s agent
Ways agents share state inside one system
- Return valuesA worker hands back a condensed result to the agent that called it.
- Hand-offThe conversation and its state move to the next agent.
- Shared threadEvery agent reads the same accumulating conversation.
- Shared storeAgents read and write named state; never let concurrent agents overwrite each other.
- EventsAgents publish and subscribe to messages carried by a durable bus.
A design, end to end
A worked design: an insurance claims intake system.
A claim arrives as an email with several attachments. The design mixes patterns, each chosen for one stage — the advice Microsoft gives for combining orchestration patterns.
- Split
Code unpacks the email into one item per attachment.
- Classify and route
A classifier labels each file; each label goes to its own extractor agent.
- Extract in parallel
Claim-form, invoice and photo agents run at once on their files.
- Join per claim
Results are gathered by claim number — only when every part has arrived.
- Check coverage
A policy agent drafts a decision; a checker agent tests it against the policy wording.
- Decide
Clear claims proceed; the rest wait for an adjuster with the evidence attached.
Where this design can break. A redelivered message counted twice at the join; a join that fires when two of three parts are back; a run marked complete while one extractor failed. The architecture must rule out each one.
Design for failure
Design rules that answer the known multi-agent failure modes.
Research on 1,600+ traces across seven frameworks found 14 failure modes in three groups: system design, misalignment between agents, and weak verification. Each rule below answers one of them.
Architecture rules
- A brief per agentRole, objective, output format, tools and when to stop — vague briefs cause duplicated work and gaps.
- Condensed returnsWorkers send back findings, not raw material, so the coordinator’s context stays small.
- Messages counted onceEvery message carries an id; a redelivery is recognised, never processed twice.
- Joins that waitCombine per key, only when every part has arrived.
- Iteration capsLoops and hand-offs stop at a set count and fall back to a person.
- One verifierA checker agent, tests or a person confirms the result before it leaves the system.
- A trace of everythingEvery message, tool call and decision, linked to the run that caused it.
On Botminds
How the Botminds agent mesh implements multi-agent system architecture.
A Botminds agent mesh is a graph you draw once: members do the work, channels say who receives what, and a durable bus carries and remembers every message. The graph is checked before it may run.
| Multi-agent framework | Botminds agent mesh | |
|---|---|---|
| Members | Agents in code | Agents, workflows, nested meshes and outside services on one canvas |
| Routing | Your code | Conditions on channels: simple comparisons joined by and/or |
| A worker crashes | Whatever recovery you build | Another worker takes the message; a keyed join recognises it |
| Joining results | Your code | Once per key, never on a partial set |
| Fan-out | As wide as one process holds | 20,000 branches measured, rebuilt byte for byte |
| Long outside work | Holds a thread | The step parks and holds no worker |
| Failures | Exceptions in a log | One cause, a severity, shown in four places — and Piper can explain them |
| Scaling | Your infrastructure | Admitted against capacity; workers and orchestrators scale out on Kubernetes |
Read the full design in the agent mesh, and build one with teams and mesh and the system pattern in the Academy.
Questions people ask
Multi-agent system architecture: common questions.
The technology behind itAgent meshMulti-agent work at internet scale→What is multi-agent system architecture?
Multi-agent system architecture is the design of a system where several AI agents with separate roles, tools and context work on one job. It covers how work is divided, how agents communicate and share state, which protocols connect them, and how runs are verified, recovered and recorded.
What are the main multi-agent architecture patterns?
Sequential (a pipeline), concurrent (fan-out and fan-in), group chat (a shared thread, including maker-checker loops), hand-off (one agent passes control to another) and magentic (a manager builds and revises a plan). Most systems combine several.
What is the difference between centralised and decentralised multi-agent systems?
In a centralised design one orchestrator or manager agent decides who works next and combines results. In a decentralised design agents hand control to each other or exchange messages over a shared bus, with no single agent in charge of the flow.
How do agents communicate in a multi-agent system?
By returning results to the agent that called them, by handing off the conversation, through a shared thread or shared state, or by publishing messages to a bus. Across vendors, A2A carries agent-to-agent tasks and MCP connects agents to tools.
What is the difference between A2A and MCP?
MCP connects an agent to tools and data: the agent calls a capability. A2A connects an agent to another agent: the client sends a task to a peer that reasons for itself. Google described A2A as complementing MCP when it launched in April 2025.
What is an orchestrator agent?
An agent that plans a task, delegates parts to other agents, and combines what they return. Anthropic’s research system uses a lead agent that spawns parallel sub-agents, then a separate pass that attributes each claim to its source.
Why do multi-agent systems fail?
Mostly in the architecture, not the models: unclear roles and stopping rules, agents withholding or ignoring information, and weak verification. Lost or duplicated messages and early joins add runtime failures that look like success.
What is an agentic AI mesh?
A term popularised by McKinsey’s QuantumBlack for an enterprise architecture in which custom and off-the-shelf agents work together under one set of controls, built on five principles: composability, distributed intelligence, layered decoupling, vendor neutrality and governed autonomy. The Botminds agent mesh is one implementation.
Sources
- Microsoft Azure Architecture Center: AI agent orchestration patternslearn.microsoft.com
- OpenAI: A practical guide to building agentscdn.openai.com
- Anthropic: How we built our multi-agent research system (June 2025)anthropic.com
- Google Developers Blog: Announcing the Agent2Agent Protocol (April 2025)developers.googleblog.com
- Linux Foundation: Agent2Agent protocol project launched (June 2025)linuxfoundation.org
- Model Context Protocol specificationmodelcontextprotocol.io
- Google Agent Development Kit: workflow agentsadk.dev
- McKinsey QuantumBlack: Seizing the agentic AI advantagemckinsey.com
- Kim et al.: Towards a Science of Scaling Agent Systems (arXiv 2512.08296)arxiv.org
- Cemri et al.: Why Do Multi-Agent LLM Systems Fail? (arXiv 2503.13657)arxiv.org
- Guo et al.: Large Language Model based Multi-Agents — A Survey of Progress and Challenges (arXiv 2402.01680)arxiv.org
Read next
See it on
your own work.
Bring one process and its documents. We’ll show you the agents, the controls and the record, running on your data.