one conversation
Agent team
A lead agent coordinates, routes to or collaborates with others inside one run. Right for a conversation that needs several specialists.
Agent mesh
The agent mesh: multi-agent work at internet scale.
Agents, workflows, nested meshes and outside services, wired as one graph — every message accounted for.
A Botminds agent mesh runs a multi-agent system as one durable pipeline. It fans work out to tens of thousands of branches, routes each message by its content, gathers the results back with a keyed join, rides through crashed workers, and carries millions of documents on workers and orchestrators that scale out on Kubernetes as load rises.
Document inan upload, an email, an event
Headsplits the work
fan-out · a channel each, in parallel
agentAgentreads and extracts
workflowWorkflowchecks the fields
meshNested mesha whole sub-pipeline
serviceOutside serviceparked, holds no worker
Keyed joina combining member gathers each key once
Answerwith its run record
One durable message bus
40,001 published40,001 delivered0 lost
a 20,000-way run, measured in our scale testsWhat is an agent mesh?
An agent mesh is an architecture for running many AI agents as one system. Each agent, workflow or service is a member; members pass messages over a shared, durable bus; and the platform — not each agent — handles routing, scaling, retries and failure. McKinsey calls the enterprise-wide version the agentic AI mesh. In Botminds, a mesh is a graph you draw once and run on millions of documents, with every message accounted for.
Scales out on KubernetesServices and workers add capacity as work arrives, and give it back when it is done.
Built for millionsMillions of documents, records and agent runs — the indexes and workers grow with them.
Any cloud, any regionAzure, AWS, Google Cloud or on-premises, in the region your data must stay in.
Always onRedundant services, rolling releases and continuous backups, for mission-critical work.
Secure and governedSingle sign-on, encryption, least privilege and a sealed audit trail.
From one agent to many
Why one agent is not enough — and why most multi-agent systems fail.
One agent with every tool gets slower, costlier and less accurate as the job grows. Splitting the job across specialists fixes that, and moves the risk into the hand-offs between them.
fixed steps
Workflow
A set sequence with an agent where judgement is needed. Right when the order never changes. See agentic workflows.
volume, durable
Agent mesh
Agents, workflows, other meshes and outside services exchange messages over a durable bus. Right for volume, fan-out and work that must survive a crash.
Lost messages
Work handed to a member that never arrives — and nothing says so.
Duplicates
A restarted worker repeats a step, and the join counts it twice.
Early joins
A combining step fires on half the parts.
False success
The run ends “done” while a branch failed silently.
Multi-agent systems rarely fail in the model. They fail in the plumbing — and each failure looks like success until a customer finds the gap. The Botminds agent mesh is built, and tested, against all four.
The agentic AI mesh
The agentic AI mesh, and what it asks of a platform.
The agentic AI mesh is a term popularised by McKinsey’s QuantumBlack: an architecture that runs custom and bought agents side by side, governed as one. Its five principles, and where each lives in Botminds:
| Principle | What it asks | In the Botminds agent mesh |
|---|---|---|
| Composability | Plug in any agent, tool or model without a rebuild. | Four member kinds — agent, workflow, nested mesh, outside service — on one canvas. A mesh can be a member of another. |
| Distributed intelligence | Split work across cooperating agents. | Fan-out to tens of thousands of branches; a combining member puts the answer back together per key. |
| Layered decoupling | Logic, memory, orchestration and interfaces change independently. | Members exchange messages over the bus. Swap an agent or its model and the mesh around it stays as it is. |
| Vendor neutrality | No lock-in to one model or vendor. | Each agent picks its own model; outside services join through a claim-and-complete contract; tools speak MCP. |
| Governed autonomy | Agents act within policies and oversight. | Tool grants, approvals and spend governance per agent; every run recorded; failures surfaced by cause. See AI governance. |
Agent mesh architecture
Agent mesh architecture: members, channels and one durable bus.
A mesh is a graph. Its members do the work; its channels say who receives what; the bus underneath carries every message and remembers it. The graph is checked before it may run, so a design mistake is a message on the canvas, not a stuck document.
workflowIntakeunpack the email, register each file
agentClassifyname each file’s kind, with confidence
Routed by content
kind == 'claim form'agentExtract the claimtyped fields against a schema
kind == 'medical bill'nested meshBill linesits own fan-out, one line per branch
kind == 'police report'outside serviceRecords checkparked until the outside system answers
combineAssemble the claimgathers until every file of the claim is in
workflowWriteto the claims collection
Checked before it may run
- one head
- no loops
- no orphan steps
- every member exists
- nesting checked
- branches that would run in turn, flagged
Agent
Any Botminds agent, with its own tools and model. It reads the message, does its job, hands on the result.
Workflow
A fixed sequence of steps — code, SQL, an API call, an agent — as one member.
Nested mesh
A whole mesh as one step, run as its own child run; the parent waits for its answer.
Outside service
Your own system. The step parks until the service claims the job and reports back — holding no worker.
Multi-agent orchestration
Fan-out, fan-in and content routing.
Multi-agent orchestration comes down to three moves: split the work, send each part to the right member, and put the parts back together correctly. The hard one is the last.
Fan-outFan-in, per key
Documentthe head splits it into pages
rebuilt, then compared byte for byte
Pageseach page into its lines
lines joined per page, in order
Lineseach line into its words
words joined per line — once per line
Wordsone branch per word, in parallel
each word handled once
Down the left, the work spreads out. Up the right, it comes back together: a combining member gathers with the platform’s keyed join, so each line, page or claim fires exactly once, never on a partial set, and a redelivered message is recognised.
Once per key
A combining member gathers with the platform’s keyed join: each line, page or claim fires exactly once, never on a partial set.
Counted once
A message redelivered after a crash carries the same id, so the keyed join recognises it.
Back in order
Children return in the order they were sent.
Any width
The loop guard counts depth, not width, so a fan-out of 20,000 runs and a loop is still caught.
classify → extract when kind == 'claim form'
classify → bill lines when kind == 'medical bill'
classify → review when confidence < 0.7
extract → assemble every message; assemble gathers per claim_idConditions are simple comparisons joined by and/or — no code runs on a channel. The same fan-out is how the platform tests itself: a document is split into words across a mesh, rebuilt, and compared byte for byte with the original.
Long work, no waste
Parked services and nested meshes: long work that holds no worker.
Some steps take minutes or hours: a records search in another system, a batch job overnight. A mesh that held a worker for each would run out of workers. In Botminds, the step parks instead.
- Park
The mesh records the job and frees the worker.
- Claim
Your service asks for its next job when it is ready, at its own pace.
- Report
It completes the job with its answer, or fails it with a reason.
- Resume
The mesh carries the answer on, as if the step had run inline.
- Named, if no one comes
A job no service picks up fails as “no worker” — a cause you can act on.
A pipeline as a step
A mesh calls another mesh as its own child run, fed the parent’s message.
Light members inline
Quick members run inside the orchestrator, with no start-up cost.
Heavy members on worker pools
Heavier members run on their own worker pools, which scale out on Kubernetes as load rises.
No lost work
Every message accounted for: delivery, recovery and dead letters.
The promise of a mesh is that no work disappears between members. It is kept by a few strict rules about each message — and by refusing to call a run finished while any of its messages is not.
- Publisheda member hands its message to the bus
- Claimedby exactly one worker; the others skip it
- Runthe next member does its work
- Donemarked done; the run’s count falls by one
A worker dies or freezes
- its claim lapses
- another worker takes the message
- same message ids, so a keyed join recognises it
Delivery keeps failing
- the bus tries it again
- then sets it aside — never drops it
- and names it on the run
The database blinks mid-publish
- the publish is retried in place
- on a fresh connection
- inside the same step
The ruleA run ends done only when every message of the run is done. Lost work never reads as success.
Recovery on purpose
A failed member fails the run with its cause. Re-running is a deliberate recovery that an operator or Piper starts.
Refused before it is sent
A member that sends to a topic it does not own is stopped at once with a named cause.
Stuck runs are found
A run where nothing is working, waiting or moving is flagged as stalled, by name.
Visibility
Failures classified by cause, seen in four places at once.
“Something went wrong” helps nobody at volume. Every failed mesh run gets one cause from a fixed list, a severity that says who should act, and a sentence in plain words — shown wherever your team already looks.
Someone must look nowcritical
- A message was lost
- The outcome cannot be read — never counted a success
The project’s to fixits input, code or settings
- A step’s own code failed
- Input the step could not take
- Content refused by the model provider
- Budget spent
- An agent or model is missing
Capacity and timeplatform or provider
- Ran out of time
- Rate limited by the provider
- Waited too long for capacity
- No worker for an outside service
- Stalled: nothing moving
Exceptions feedone group per mesh and cause, with sample runs
Action Centeran item that names the cause, for failed and stalled runs
The mesh’s Runs tabfilter to failed; the message trail marks what was lost
Ask Piper“what broke today?” — answered in plain words
Asking Piper what broke
You
Which meshes failed today, and why?
Agent
Two causes in the last 24 hours. Claims intake: 14 runs ran out of time at the records check — the outside service had no worker after 6 pm. Invoice lines: 3 runs failed because a step’s own code raised on a credit note with no lines. Nothing is stalled, and no messages were lost.
You
The records service is back. Re-open those 14 runs.
Agent
Re-opening restarts each from the records check; the steps before it are kept. Re-open 14 runs?
Scaling AI agents
Scaling AI agents: the agent mesh at internet scale.
The agent mesh is built for internet-scale workloads: millions of documents flowing through meshes, durable messages, priority lanes, and workers and orchestrators that scale out on Kubernetes as load rises — on Azure, AWS, Google Cloud or on-premises, in any region.
Millions of documents
- Uploads
- Connectors
- Your API
- Events
Admitagainst real capacitymodels · database · workers
Durable message bus
Express
Normal
urgent sets go first; normal work keeps its reserved shareWorkers · scale out on Kubernetesload rises, capacity is added; it falls, capacity is given back
- Durableevery message stored until it is done
- Any worker, any messagea lost worker’s claim passes to the next
- Rolling releasesnew versions roll out while the work flows
- Any cloud, any regionAzure, AWS, Google Cloud or on-premises
Millions of documents
Every document is a run; every run fans out. Volume is served by adding workers, so the same mesh carries a pilot and a whole enterprise.
Scale out on Kubernetes
Workers and orchestrators scale out on Kubernetes as load rises and give capacity back as it falls; document workers follow the depth of their queue.
Admit, then scale
Each document is admitted against the real capacity of the models, the database and the workers, with a plain verdict: what runs, what waits, and why.
Priority lanes
Turn on priority for a mesh, then mark a document set urgent at any time — by hand or through Piper. It goes to the front, while normal work keeps its reserved share.
Measured in our scale tests
measured20,000parallel branches from one document, rejoined byte for byteTen times the width, 9.5× the time — a straight line, not a curve.
measured0messages lost
- 40,001 published
- 40,001 delivered
- 160/160 documents exact, through eight mesh shapes
measured4.5×the work, on four processes instead of onelinear · every process added grows the total
Break a worker in the middle of the work
FaultResult
Killeda worker process killed outright, mid-wave, not restarted
20/20
Cut offevery database connection of one worker terminated at once
20/20
Restarteda worker restarted in the middle of the wave
20/20
Frozena worker frozen for 40 seconds, then let go
20/20
one document, finished exact, byte for byte
Side by side
Agent mesh vs multi-agent framework vs workflow engine.
All three can put several agents to work on one job. What separates them is what happens at volume, and on a bad day.
| Multi-agent framework | Workflow engine | Botminds agent mesh | |
|---|---|---|---|
| What it is | A library for agents that talk in code | A scheduler that runs fixed steps | A running system of agents, workflows, meshes and services |
| A worker crashes | Whatever recovery you build | The step is retried; duplicates are yours to catch | Another worker takes the message; a keyed join recognises it |
| Fan-out | As wide as one process holds | Branches drawn by hand | 20,000 branches measured, byte-exact |
| Joining results | Your code | Wait for all steps | A keyed join: once per key, never on a partial set, redelivery recognised |
| Long outside work | Holds a thread | Holds a worker | Parks; holds nothing |
| Failures | Exceptions in a log | Step status | One cause, a severity, four places, and Piper |
| Scaling | Your infrastructure | Add workers by hand | Admitted against capacity; workers and orchestrators scale out on Kubernetes |
Already using a framework? Keep it for the agent logic and bring the agent in as an outside service member, or rebuild it as a Botminds agent. See Botminds vs CrewAI and the build vs adopt whitepaper.
Questions people ask
Agent mesh, answered.
More from the platform: the full architecture, and how security works at Botminds.
What is an agent mesh in AI?
An agent mesh is an architecture for running many AI agents, workflows and services as one system. Members exchange messages over a durable bus, and the platform handles routing, joins, scaling, retries and failure for all of them, so no agent has to.
What is the agentic AI mesh?
The agentic AI mesh is a term popularised by McKinsey’s QuantumBlack for an enterprise architecture that runs custom and off-the-shelf agents together under one set of controls. Its principles are composability, distributed intelligence, layered decoupling, vendor neutrality and governed autonomy.
What is a multi-agent system?
A multi-agent system splits a job across several specialised agents instead of one general one. It is more accurate and cheaper per task — as long as what carries the work between agents is reliable, which is what a mesh provides.
Why do multi-agent systems fail?
Mostly in the hand-offs, not the models: a message lost between agents, a step repeated after a restart, a join that fires on half the parts, or a run marked finished while a branch failed. The Botminds mesh is built and tested against each of these.
What is the difference between an agent mesh and multi-agent orchestration?
Orchestration decides which agent runs next. A mesh is the running system that carries the work between them durably — fan-out, joins, delivery, failure and scale. In Botminds, agent teams orchestrate inside a conversation; the mesh carries work across documents and machines.
How does an agent mesh scale?
By adding workers. Workers and orchestrators scale out on Kubernetes as load rises and give capacity back as it falls; document workers follow the depth of their queue. Every document is admitted against the real capacity of models and database first. In our scale tests, four processes did 4.5 times the work of one, and one document fanned out to 20,000 branches came back byte for byte.
How many documents can an agent mesh process?
The mesh is built for millions of documents. Volume is served by adding workers and orchestrators, which scale out on Kubernetes on any cloud and region; urgent sets jump the queue while normal work keeps its share.
What happens if a worker crashes in the middle of a run?
Its claim on the message lapses and another worker takes it, with the same message ids, so a keyed join recognises it. In our scale tests, with a worker killed, cut off from the database, restarted or frozen mid-wave, every document came out exact every time.
Can a mesh call my own systems?
Yes. Register your system as a service: the mesh parks the step, your service claims the job when ready and reports the answer or a failure. Agents in the mesh can also reach your systems through tools and MCP.
How do I know when a mesh fails?
Each failed run gets one cause from a fixed list and appears in the exceptions feed, the Action Center and the mesh’s Runs tab. You can also ask Piper what broke in plain words. See evaluation and observability.
Bring your heaviest pipeline. We’ll run it as a mesh.
Show us the work that crosses five teams and three systems. We’ll show you it as one mesh — fanned out, joined back, scaled out, and every failure named.