The Pilot Looks Perfect Until It Does Not
A distribution client ran a document extraction agent for six weeks in pilot. Thirty invoices a day, near-perfect accuracy, operations team loved it. They approved full rollout. On day two of production the agent was processing four thousand invoices and the whole thing was on the floor by noon. Not because the model got worse. Because nothing in the pilot had any reason to surface what volume actually does to an agent.
That is the core problem. Pilots are quiet. Production is loud. And the gap between ten runs a day and ten thousand is not a scaling problem in the software-engineering sense. It is a category change. The failure modes are different, the costs are different, and the fixes are different. If you have shipped a promising pilot and are now planning rollout, this is what you are about to run into.
Rate Limits Are Not a Minor Annoyance at Scale
At ten runs a day you will never see a rate limit. At ten thousand you will see them constantly, and if your agent is not built to handle them gracefully, you will see cascading failures instead of graceful degradation.
The specific shape of this problem in production is a retry storm. An agent hits a rate limit, retries immediately, hits the limit again, retries again, and now you have multiplied your request volume at exactly the moment you are already over quota. I have watched a manufacturing client's agent triple its API call volume in four minutes because the retry logic had no backoff and no jitter. The fix sounds simple in retrospect: exponential backoff with randomized jitter and a hard cap on retry attempts. But you will not write that logic until you have been burned, because in a pilot it never matters.
The less obvious rate limit problem is that limits are often tiered by endpoint, by model, by organization, and sometimes by time of day. An agent that calls three different upstream services will hit three different limit profiles simultaneously under load. You need to track each one independently, not treat rate limiting as a single on-off switch.
Cost Curves Bend in Ways Your Spreadsheet Did Not Predict
Pilots are almost always cost-positive on paper. You run the math, you multiply unit cost by expected volume, you get a number that looks fine. What the spreadsheet misses is that agents under load do not behave like agents in a demo.
Three things drive cost up faster than expected at scale. First, error handling. When a step fails and the agent retries, you pay for the retry. When the retry includes re-sending a long context window because the agent needs to re-establish state, you pay a lot for the retry. In a financial services workflow I worked on, roughly twelve percent of total token spend was pure retry cost once we measured it properly. That was invisible at low volume.
Second, context bloat. Agents that accumulate conversation history or tool call results in a growing context window get more expensive per run as time goes on, not just as volume goes up. A run that costs X tokens at step three costs three times X at step nine. At ten runs a day that curve is invisible. At ten thousand it is a budget problem.
Third, fan-out. An agent that spawns sub-agents or parallel tool calls under certain conditions might do so rarely in a pilot. Under high volume those rare conditions become common. A field operations agent that I helped stabilize was spawning an average of 1.4 sub-calls per run in testing. In production it was 3.1, because the edge cases that trigger fan-out are more common in real data than in curated test sets. That 2.2x multiplier was not in anyone's cost model.
Concurrency Exposes Every Assumption You Made About State
Single-threaded agents hide a class of bugs that only appear when multiple instances run simultaneously. This is the one that surprises engineers who have built reliable software for years, because the mental model for agents is often sequential even when the deployment is not.
The most common version of this in my experience is shared mutable state. An agent writes intermediate results to a cache or a database row, and under concurrent load two instances read stale data, both decide to act on it, and you get duplicated actions or conflicting writes. In a manufacturing context this produced duplicate purchase orders. Not often, but the ones that got through were expensive to unwind.
The second concurrency problem is ordering assumptions. An agent designed to process items in sequence may have implicit logic that assumes item B comes after item A. At scale, item B might arrive at the processing layer before item A finishes. I have seen this produce downstream reports that were internally inconsistent because the agent was aggregating across a partially-complete state.
The fix is to treat each agent run as fully isolated by default. No shared mutable state between concurrent instances. If you need coordination, use explicit locking or queuing, not implicit assumptions about timing. This is not a new idea in distributed systems. It is just not how most people prototype agents.
The Failure Modes That Only Appear in Real Data
Pilots run on clean data, or at least cleaner data than production. Real data at volume is messier, more varied, and full of edge cases that nobody thought to include in a test set.
The most operationally damaging version of this is silent degradation. The agent does not error out. It returns a result that looks plausible but is wrong in a way that is hard to catch without domain knowledge. At ten runs a day a human reviewer might catch this. At ten thousand runs a day it is in the downstream system before anyone notices. In a finance workflow, this showed up as correctly-formatted but semantically wrong categorizations that passed all automated validation and only surfaced in a monthly reconciliation.
A related failure mode is prompt sensitivity under distribution shift. The instructions that work well on the data you used to develop the agent may work less well on the full range of data in production. This is not the model hallucinating in some dramatic way. It is the model making subtly different choices when the input looks different from what the prompt was implicitly tuned against. You will not see this in a pilot. You will see it at scale when the long tail of unusual inputs starts to matter.
The practical response is to build evaluation into the pipeline, not just before deployment. You need a sample of production outputs reviewed against ground truth on a regular cadence, with metrics that can tell you whether quality is drifting before the drift becomes a business problem.
Infrastructure Choices That Do Not Matter at Ten Matter a Lot at Ten Thousand
Synchronous request handling is fine for a pilot. At scale it means a slow upstream response blocks a thread and your throughput collapses. Every production agent running at meaningful volume needs an async execution model with a proper queue, not a direct call chain.
Observability is the other one. At ten runs a day you can read logs. At ten thousand you cannot. You need structured logging from the start, with run IDs that propagate through every step and every sub-call, so that when something goes wrong you can reconstruct exactly what the agent did and why. I have spent days debugging production incidents that would have taken an hour with proper trace IDs. The agents that are easiest to operate at scale are the ones where someone made observability a first-class requirement before rollout, not an afterthought after the first incident.
Timeout handling deserves its own mention. An agent that waits indefinitely for a slow tool call will eventually accumulate enough hung threads to take down the whole worker pool. Set timeouts on every external call. Decide in advance what the agent should do when a tool call times out, because it will time out, and the answer should not be to wait forever.
The Short Practical Takeaway
Before you scale a pilot to production volume, do four things.
- Run a load test at two times your expected peak, not your expected average, and watch what happens to retry rates, cost, and error distribution.
- Audit every place your agent touches shared state and make concurrent access explicit, not assumed.
- Build a sampling-based evaluation loop that runs continuously in production, not just before launch.
- Set hard timeouts and budget caps at the infrastructure level, not just in the application code, so a runaway agent cannot take down adjacent systems or generate an unexpected bill.
None of this is exotic. It is the same discipline that good distributed systems engineering has required for twenty years. The reason it catches people off guard with agents is that agents feel like software products during development and feel like infrastructure problems in production. The sooner you treat them as infrastructure, the fewer surprises you will have when the volume arrives.