For most of the history of enterprise technology, the bottleneck was the same. Engineering teams couldn’t build at the pace at which the business came up with new ideas. AI has inverted that dynamic. The technology now moves faster than the organization around it can absorb, and the result is a widening gap between what AI can do in a demo and what it can sustain in production.
That pressure point was the focus of a webinar featuring four practitioners who’ve operated AI at scale across retail, e-commerce, and SaaS. Colin Bodell, former CTO at Groupon, and American Eagle Outfitters hosted the panel formed by Bill Groves, Senior Partner at Bain & Company and former Chief Data & Analytics Officer at Walmart, Krishna Vedula, Head of Technology, Product, and Operations at WellnessLiving, and Rajesh Rudraradhya, CTO at Lytx and former CTO at Snapdeal.
The panel’s backgrounds are rooted in retail and e-commerce, where the pilot-to-production gap tends to surface fastest. Inventory, fraud detection, customer experience, and fulfillment all operate on tight feedback loops. When an AI system underperforms in production, the revenue impact is immediate and visible, but the patterns the panel identified are structural, not industry-specific.
The panel covered several topics including why the business can’t keep pace with the technology, the operational investment production actually demands, the architectural discipline agentic systems require, and the organizational alignment without which none of it scales.
The Business Isn’t Ready for What the Technology Can Already Do
Bill Groves called out the big issue early in the discussion. For the first time, technology is outpacing the business rather than the other way around. Historically, the constraint was always engineering capacity; the business had ideas but couldn’t get them built fast enough. AI has flipped that. The models, the agents, and the tooling are ready. However, the business processes and governance structures are not. Most organizations aren’t used to being in that position.
That plays out in a specific pattern. Teams build impressive prototypes to demonstrate value quickly, and then deploy into production environments that can’t absorb the speed or the consequences. Groves recalled his time at Walmart. When online grocery launched, shelves emptied overnight because inventory systems weren’t designed to handle simultaneous online and in-store demand. The technology worked. The operational infrastructure behind it didn’t. The result was one of the worst possible retail outcomes. Customers walked into a store to find nothing on the shelf.
But the readiness problem isn’t only about infrastructure. When the business hasn’t caught up, there’s no clear signal telling teams what to aim at, and Krishna Vedula argues that’s when they start chasing the wrong targets entirely. “Are you defining the right outcome, or are you chasing that rush of being able to use AI for something cool?” he asked. “I would strongly advocate against chasing that dopamine rush as opposed to solving that business outcome.”
The distinction between outputs and outcomes ran through the entire panel. An AI feature that generates a neat report or automates a screen is an output. An AI system that increases basket size, or reduces friction for a customer is an outcome. Groves was blunt about incremental work that doesn’t clear that bar. He claims that it doesn’t do anything but spend money. The test for whether an AI investment is ready for production starts with whether the business has defined what success actually looks like.
Production Readiness Is the Work Few are Budgeting For
We talked about the business not being ready, and the natural question that follows is, what does readiness actually require? Rajesh Rudraradhya put a number on it. The operational investment needed to sustain AI in production can run twice what it takes to ship a traditional SaaS system. Most organizations drastically underestimate that because the prototype phase feels so effortless.
“All of you are over-investing in model capabilities, which is necessary for the first phase,” he said, “but the operational readiness part of it is much, much larger than what you’ve had to build traditionally.”
His team learned that through a series of concrete production failures. Model providers that were assumed to have near-perfect uptime went down, and the product had no fallback path. Real-time features built around AI models hit unpredictable latency that broke SLAs. Agents that performed well in staging behaved differently at production scale. And multi-agent orchestration systems stalled at handoff points because the business context governing each agent hadn’t been defined precisely enough. All of these were system design problems, the kind engineers have solved before, but that get overlooked when the prototype looks so promising.
The risk extends to customer-facing systems, too. Krishna described a scenario where a high-value item is lost in transit. A well-designed AI system detects the delay, notifies the customer, and offers a replacement. A poorly designed one creates friction or losses without anyone realizing until the customer’s complaint arrives. “Be deliberate, be thoughtful about how you insert AI into your operations,” he said.
Rajesh identified a pattern that compounds the risk. During prototyping, the strongest engineers are on the project because proving value quickly is the priority. When the system moves to production, those engineers get pulled to build the next agent. The team left to operate and govern the system in production doesn’t fully understand what was built or what it was designed to do. “That’s why a lot of these things fall apart in production,” he said, “because your best team is off building the next agent.”
The implication is a resource planning shift that many organizations haven’t made. Production isn’t the finish line for AI systems. It’s the point where the real operational work begins, and team composition has to reflect that.
Agentic Systems Are Unbounded by Default and Architecture Has to Fix That
The panel described what goes wrong when the organization and the operational investment aren’t there. But even when teams staff correctly and budget for production, agentic AI introduces a category of architectural risk that traditional systems never had to account for.
Rajesh put it simply, “What this really does to you is it makes your system unbounded in many different situations. For all of you that have built large-scale systems, you don’t want your systems to be unbounded.”
To understand why that matters, consider how software has worked until now. Traditional systems are largely predictable. You can trace the paths through the code, test them, and predict how the system will behave at scale. Agentic AI is non-deterministic by design. Agents interact with other agents, including external ones that your team doesn’t control. Latency varies and costs are unpredictable. A behavior that passed every test in staging can diverge in production because the conditions themselves keep shifting.
Colin Bodell raised the additional layer of multi-agent environments, where your system is interacting with agents run by other companies, each optimizing for their own objectives. Thus, observability challenge alone extends well past the boundaries of your own infrastructure.
Groves pointed to fraud detection as a practical example. When agentic bots start shopping alongside human customers, purchasing patterns change. Transactions that existing fraud systems are trained to catch today will look different tomorrow, and protections the business already relies on quietly degrade without anyone updating them.
Rajesh’s response to that complexity was deliberately pragmatic. He advised to think through every way the system can fail, and have a plan for each one. “It’s not like it’s going to fail. It is going to fail. The question is how well do you recover,” he said.
Krishna approached the bounding problem from the other direction. Where Rajesh’s answer is to plan for every failure, Krishna’s is to limit where you introduce that complexity in the first place. Not every task needs the most powerful model. Not every workflow needs an agent. “I think it’s time to move away from token maximization to token optimization,” he said, and proposed framing AI as “a tool belt, as opposed to it being a single tool.” His point was that simple tasks should stay with straightforward automation, and agentic approaches should be reserved for decisioning use cases where the complexity and the operational overhead are justified.
Alignment Isn’t a Soft Problem. It’s the Scaling Bottleneck
As seen through different situations, the business can’t keep pace with the technology. Production demands more operational investment than organizations expect. And the architecture itself requires new discipline to stay bounded. But Bill Groves argued that none of those problems get solved if the organization isn’t aligned on what it’s doing and why.
When an audience member asked the panel to name the biggest bottleneck to scaling AI, Groves moved past his own earlier answer about data foundations. “I believe for almost all technology, not just AI, organizational alignment and operating models are typically the biggest reason things fail,” he said. “You have the best data in the world, but if you don’t have the right operating model, the executives aren’t working well together, picking out which areas to focus on… it really doesn’t matter what tech you build.”
His argument throughout the webinar was that the traditional separation between business and technology teams has become a liability. “I shouldn’t be able to tell the difference between who’s my engineer, who’s my product manager, and who’s my category manager,” he said. “They should be working so close together I shouldn’t know the difference.”
When outcomes aren’t clearly defined, engineers build what’s technically interesting. When AI decisions run through siloed teams, production readiness gets underscoped because nobody owns the full picture. The alignment problem sits beneath every operational and architectural failure the panel described.
The Technology Is Ready. The Organization Has to Be, Too
The panel’s diagnosis was consistent across all four themes. What stalls AI is the operational investment, architectural discipline, and organizational alignment that production demands and prototyping doesn’t. Engineers have solved these categories of problems before. What’s changed is the speed, and how far the consequences travel when something goes wrong.
BairesDev partners with organizations to navigate his transition, embedding AI development teams that bring the production discipline agentic systems require. If your team is moving from pilot to production and the gap between the two feels wider than expected, reach out to talk about it.
Watch the full webinar here.
