The Mistake Happens at the Whiteboard
Most teams that build multi-agent systems when they did not need to make the decision in the first meeting, before they have written a single line of prompt or run a single test. Someone draws boxes on a whiteboard, each box gets a name like Orchestrator or Validator or Retrieval Agent, and the architecture feels smart before it has done anything. That feeling is expensive.
I have seen this pattern in every sector I work in. A distribution company wants to automate freight exception handling. They sketch five agents. Three months later they are debugging message passing between agents that each work fine in isolation but fail at the seams. The actual task, classify the exception, look up the contract rule, draft the carrier message, could have run in one agent with three tool calls. One agent would have been in production in six weeks.
The question is not whether multi-agent is powerful. It is. The question is whether your specific task actually requires it.
What a Single Agent Can Actually Handle
A well-scoped single agent with good tools and a clear system prompt can handle more than most people expect. The ceiling is higher than the whiteboard crowd thinks.
In manufacturing I have run single agents that pull sensor anomaly data, cross-reference maintenance logs, draft a work order with the right part numbers, and route it to the right supervisor queue. That is four distinct actions across three systems. It is not a simple task. But it is a linear task with predictable branching, and one agent handles it cleanly. The context window holds everything it needs. There is one place to look when something goes wrong.
In field operations, a single agent handling technician dispatch can check availability, read the job history for that site, apply scheduling rules, and write the dispatch note. Again, multiple steps, one agent, one log to read when the output is wrong.
The honest rule is this. If you can describe the full task as a sequence of steps where each step depends on the one before it and the total context fits comfortably in a single window, you probably do not need more than one agent. Add tools, not agents.
Three Conditions That Actually Justify Multiple Agents
There are real cases where splitting work across agents earns the overhead. I have shipped these systems. Here is what the legitimate cases look like.
- The task has genuinely parallel workstreams that do not depend on each other. In a financial reconciliation workflow I built, one agent was pulling and normalizing transaction data from one system while a second was doing the same from a counterparty system. Neither needed to wait for the other. Running them in parallel cut wall-clock time by more than half. That is a real gain, not an architectural preference.
- The context required for one part of the task would pollute or crowd out the context needed for another part. In a complex insurance underwriting assist tool, the agent doing document extraction needed deep context about document structure and extraction rules. The agent doing risk scoring needed actuarial logic and product rules. Cramming both into one context made both worse. Separating them made each one sharper and easier to evaluate independently.
- You need different reliability or escalation behavior in different parts of the workflow. In a distribution center returns processing system, the classification agent could tolerate a higher error rate because a human reviewed low-confidence outputs. The inventory update agent needed much tighter guardrails because its outputs wrote directly to a system of record. Separate agents meant separate confidence thresholds, separate fallback logic, and separate monitoring. That separation was worth the complexity.
The Coordination Tax Is Real and Underestimated
Every agent boundary you introduce is a place where information can be lost, misformatted, delayed, or misrouted. Every handoff is a potential failure mode that does not exist in a single-agent design. This is not a theoretical concern. It is the thing that eats your debugging hours.
When a multi-agent system produces a wrong output, your first question is always which agent caused it. Was it the input to the orchestrator, the instruction passed downstream, the tool call in the subagent, or the way the result was interpreted on the way back up? In a single-agent system that question has one answer. In a four-agent system it has many.
I budget roughly two to three times the evaluation and monitoring effort for a multi-agent system compared to a single-agent system doing equivalent work. That is not a reason to never build multi-agent. It is a reason to make sure the gains you get from the architecture are real enough to justify that budget. If you cannot name the specific gain you are getting from each agent boundary, remove the boundary.
How to Run the Decision in Practice
Before any architecture conversation, I ask the team to write out the task as a plain numbered list of steps. Not a diagram. A list. This forces precision.
Then I ask four questions in order.
- Can one agent with the right tools complete this list without the context becoming unmanageable? If yes, start there.
- Are any steps genuinely parallel and time-sensitive enough that running them sequentially would hurt the outcome? If yes, those steps are candidates for a separate agent.
- Does any step require such specialized context that including it in a general system prompt would degrade the quality of other steps? If yes, that step is a candidate for a separate agent.
- Do different steps need meaningfully different reliability guarantees or escalation paths? If yes, separation may be worth it.
If the answer to questions two through four is no, you build one agent. You add tools. You write a tighter system prompt. You instrument it well. You ship it.
If the answer to any of two through four is yes, you draw one boundary, not five. You build the minimum number of agents that solves the specific problem you identified. You do not pre-architect for complexity you have not yet encountered.
What Operational Teams Get Wrong About Orchestrators
The orchestrator pattern, one agent that directs other agents, is genuinely useful when you have a dynamic task where the right next step depends on what previous steps returned. In financial audit workflows where the findings in one document determine which other documents need to be pulled, an orchestrator makes sense. The branching is real and data-driven.
But I see teams reach for orchestrators when what they actually have is a fixed workflow with a few conditional branches. A fixed workflow with conditional branches is not a reason to build an orchestrator. It is a reason to write better conditional logic in your single agent or in the application layer around it.
Orchestrators add value when the branching is genuinely unpredictable at design time. They add overhead when you are using them to manage complexity you introduced by splitting a simple task into too many pieces.
The other mistake is building an orchestrator before you have validated that the subagents work reliably on their own. An orchestrator that coordinates unreliable subagents does not fix the reliability problem. It hides it and makes it harder to find.
The Practical Takeaway
Start with one agent. Give it good tools, a clear scope, and honest evaluation. Ship it. Learn what it gets wrong. The failure modes you discover in production will tell you whether you need more agents and exactly where the boundaries should go. That is better information than anything you will generate at a whiteboard.
Multi-agent systems are not more sophisticated than single-agent systems in any way that matters to your users. They are more complex to build, debug, and monitor. That complexity is worth paying when the task genuinely requires parallelism, specialized context separation, or differentiated reliability guarantees. It is not worth paying to feel like you are doing serious AI engineering.
The teams I have seen ship the most reliable production agents are the ones who treated adding an agent boundary the same way a good engineer treats adding a service in a distributed system. You do it when you have a specific reason, you know what problem it solves, and you are prepared to own the operational overhead it creates. Not before.