The Argument Nobody Wants to Hear
In three years of shipping AI agents into production across finance, manufacturing, distribution and field operations, I have never once had a project fail because we picked the wrong model. Not once. The model debate, which foundation model, which version, which provider, consumes enormous energy in enterprise AI conversations and it is almost entirely the wrong conversation to be having.
The real blockers are unglamorous. They are data pipelines that were never built for machine consumption. They are evaluation frameworks that do not exist, so nobody can prove the system is working. They are ownership questions that nobody answered before the build started. They are integration points that look simple on a whiteboard and turn into six-month ordeals in practice.
If your AI project is stalled or underperforming, I will give you very high odds that one of those four things is the actual cause. This article is about how to diagnose which one and what to do about it.
Data Access Is the First Thing to Audit
A distribution company I worked with spent four months building an AI agent to automate freight exception handling. The model worked fine in testing. In production, the agent had read access to one system and needed data from three others. Nobody had sorted out the permissions, the API contracts or the data formats before the build started. The agent was technically impressive and operationally useless for another three months while the data plumbing got sorted out.
This is the single most common failure pattern I see. Teams treat data access as an infrastructure detail to handle later. It is not. It is the first thing to resolve, before you write a single line of agent logic.
The questions to answer before you build anything are straightforward. What systems does this agent need to read from and write to. Who owns access to those systems and what is the approval process. Are the data formats consistent enough to be useful or will you need a normalization layer. What is the latency on that data and does it match what the agent actually needs to operate.
In a manufacturing context I worked in, the answer to that last question was brutal. The agent needed near-real-time sensor data. The data was being batched and written to the warehouse every four hours. The agent was not broken. The data architecture was incompatible with the use case. Fixing that took longer than building the agent.
Without Evaluation You Are Flying Blind
The second blocker is the one that makes me most frustrated because it is entirely avoidable. Teams build AI systems without defining what good looks like. Then they cannot tell whether the system is working, cannot defend it to stakeholders, and cannot improve it systematically. They end up in a situation where someone has a bad experience with the output and the whole project gets questioned, because there is no data to push back with.
In a financial services firm I worked with, an AI agent was processing loan document summaries. It had been running for two months. When I asked the team how they knew it was performing well, the answer was that nobody had complained. That is not evaluation. That is hope.
A real evaluation framework for a production AI system has three components. First, a golden set, a collection of inputs with known correct outputs that you can run the system against regularly. Second, a set of metrics that actually map to business outcomes, not just model accuracy scores. In the loan document case, the right metric was whether a human reviewer needed to correct the summary before it moved forward, not whether the model was confident. Third, a feedback loop that captures real-world outcomes and feeds them back into your evaluation process.
Building this takes time. It takes subject matter experts who understand what correct looks like. It is not exciting work. It is the work that determines whether you can actually operate and improve the system over time, or whether you are just hoping it keeps working.
Ownership Gaps Kill More Projects Than Bad Models Do
AI agents in production touch multiple systems, serve multiple teams, and produce outputs that affect real decisions. The question of who owns the agent, who is responsible when it produces a bad output, who approves changes to its behavior, and who has authority to shut it down, needs to be answered before you go live. In my experience, it almost never is.
I have seen this play out in field operations work. An agent was routing field service technicians based on job priority, location and skill match. It was working well by any technical measure. Then it made a routing decision that a regional manager disagreed with. Nobody knew whose call it was to override the agent, whether the override should be logged, or whether the agent's logic should be adjusted. The argument that followed was not about AI. It was about organizational authority that had never been clarified.
The practical fix is to treat the AI agent like any other operational system with a clear owner. That owner is accountable for performance, responsible for communicating changes, and empowered to make decisions about the system's behavior. In most enterprise environments this means one named person, not a committee, not a shared team inbox.
The owner also needs to be someone close enough to the operational reality to know when the agent is producing outputs that look correct but are wrong in context. Models can produce plausible-sounding outputs that are wrong in ways that only a domain expert would catch. That is not a model problem. It is a monitoring and ownership problem.
Integration Is Where Timelines Go to Die
The fourth blocker is integration, and it is the one that most consistently destroys project timelines. Not because integration is technically hard, though sometimes it is, but because the systems that enterprise AI agents need to connect to were built by different teams, at different times, with different assumptions, and nobody has a complete picture of what it actually takes to connect them.
In a manufacturing environment, I worked on an agent designed to flag procurement anomalies. The agent needed to pull from an ERP system, a supplier portal, and a contract management tool. The ERP had a well-documented API. The supplier portal had an undocumented API that a contractor had built five years earlier and nobody fully understood. The contract management tool had no API at all and required a workaround through a reporting export. What looked like a three-week integration task took four months.
The lesson I take from that and from similar situations is to do integration discovery before scoping the project, not during the build. That means sitting down with the teams who own each connected system, understanding the actual state of their APIs and data exports, identifying the authentication and authorization requirements, and building a realistic estimate of integration effort based on what is actually there, not what the architecture diagram suggests should be there.
It also means being honest with stakeholders about what integration complexity means for timelines. The pressure to compress timelines is real. Giving in to it without surfacing integration risk is how projects end up three months late and over budget.
What to Fix First
If you are trying to prioritize, here is the order I use when I am brought into a stalled project.
- Data access first. If the agent cannot reliably get the data it needs, nothing else matters. Audit every data dependency, confirm access, confirm formats, confirm latency. Do not build on assumptions.
- Evaluation second. Define what good looks like before you go further. Build a golden set. Agree on metrics that map to business outcomes. Without this you cannot demonstrate value and you cannot improve the system.
- Ownership third. Name the owner. Define the escalation path. Clarify what happens when the agent produces an output that someone wants to override. Get this in writing before go-live.
- Integration last in the sense that you tackle it after the above, but early in the actual build sequence. Do integration discovery before you scope the project. Surface the complexity before it surprises you mid-build.
The model question, which one to use, whether to fine-tune, whether to switch providers, is almost always downstream of these four. Fix the fundamentals and you will find the model is performing better than you thought. Ignore the fundamentals and no model upgrade will save you.
The Practical Takeaway
The next time someone in your organization proposes switching models because the AI project is not delivering, ask four questions first. Do we have reliable, consistent access to the data the agent needs. Do we have a defined evaluation framework with metrics tied to business outcomes. Is there a named owner who is accountable for this system's performance. And do we have a clear picture of what it actually takes to integrate with every connected system.
If the answer to any of those is no, that is your problem. Fix it before you touch the model. In my experience, fixing those four things resolves the performance gap in the majority of cases without any model change at all.
AI projects stall because of organizational and operational gaps, not because the underlying technology is not good enough. The technology is good enough. The question is whether the environment around it is ready to support a production system. That readiness is built through unglamorous work on data, evaluation, ownership and integration. It is the work that separates systems that ship and keep running from systems that demo well and quietly die.