Executive Summary
This article argues that AI-accelerated development compressed the preparation window that long project timelines provided as a byproduct. The constraint has shifted from building software to absorbing it operationally. The author presents a learning loop framework and five operational decisions engineering leaders should make before launch so their organizations are ready to learn from live volume once it arrives.
Engineering organizations are absorbing a new generation of tooling. Coding agents like Cursor, Claude Code, and Codex have moved from experiments to the default way many teams write software, and delivery timelines are shrinking. In leading organizations, work that took six months a year ago ships in six weeks. Most engineering leaders are treating that compression as pure gain, and inside product development, it largely is.
Outside the software development cycle, that compression costs leaders something few have replaced. A six-month project was also six months of preparation. Prudent leaders piloted new systems next to the proven ones, took them live one region at a time, and found the edge cases while there was still a team in place to respond. Leaders planned for training, parallel deployment, and the hypercare period after go-live, and the schedule fit all of it. That was the preparation window. Compressing the project to weeks closes it. The work arrives all at once, at full volume, and the readiness that used to be a byproduct of a long project is now something leaders have to build on purpose.
AI raised the stakes on an old problem. Even with ordinary software, a team can stand up a new capability in weeks, and the rest of the organization then needs months to adopt it, staff it, and integrate it into the operation. The constraint is adoption, and the faster the build gets, the wider that gap opens.
The same pattern played out when electricity reached the factory. Factories that bolted electric motors onto the old central shaft waited decades for the productivity gains. The ones that gave every machine its own motor and redesigned the floor around the work got them.
Engineering leaders inherit the factory owner’s problem. Preparing the organization to absorb what you ship has become part of the job, and the teams that grasp it are pulling ahead of the ones still optimizing development speed.
The Constraint Just Moved Out of Engineering
Last year, I sponsored an innovation sprint for an insurance company. The company grew by taking integrations it had already built with its partners and rolling them out in new states, each with its own rules. The software barely changed from one state to the next. The team reused the APIs, workflows, and patterns that had worked in the last one. Each rollout still took two to two and a half months.
When the company’s leadership traced where the time went, roughly 70 percent of the effort sat in two places. One was defining what the new state required. The other was getting the partner to sign off on the result. Deciding what to build, and proving it worked, cost more than the building itself. The constraint had moved into the business side of the rollout, where faster engineering could not touch it.
For years development was the bottleneck, so that is where the attention, the tooling, and the people went. That made sense, but it no longer describes the situation. Once the constraint moves, Eliyahu Goldratt’s Theory of Constraints takes over. He stated that speeding up a step that is no longer the constraint does not make the system faster. It piles work up in front of the step that is.
A long development cycle was doing two jobs at once. It produced the system, and it bought the team time to learn how the system behaved before the system mattered. The first job compressed to almost nothing. The second did not come along for the ride. The learning still takes the time it always took.
Business Processes Have No Equivalent Test Suite
The learning cannot be compressed for a reason familiar to every engineer. When a suite runs, the verdict comes back green or red, and readiness is a fact you can check. Most business processes have nothing comparable.
The missing test suite resets what “done” means. A service that passes its tests can fairly be called done. An AI workflow that passes its demo cannot, because the demo runs on inputs the team selected and production runs on whatever arrives.
Suppose a purchase order comes in for approval. The vendor’s price has drifted since the contract was signed, and the buy is small enough that the ERP’s matching rules never check it. Nobody ever wrote a tolerance for this case, so nothing flags it. The workflow will either approve a price nobody agreed to or hold the order in a queue nobody is watching. A person who knows the contract has to decide what should happen and write the decision down so the next one resolves faster.
The insurance rollouts needed both kinds of testing. Automated tests covered the code. Confirming that an integration met a new state’s rules was judgment work, carried out case by case, and a large share of that 70 percent went to it.
AI workflows push the same problem further. A workflow run twice on the same input can return two different answers, depending on context, model version, and the data available at the moment it runs. You cannot prove a system like that correct in advance. All you can do is watch it run and learn from what it does. The learning the preparation window used to supply comes only from live volume and the variation it brings. Upfront architecture cannot substitute for it.
The Learning Shows Up as Exception Volume
Every case a workflow cannot handle needs to go to a person. Those cases are the exceptions, and the arithmetic behind their volume is worth doing before launch.
A workflow that clears 90 percent of 1,000 pilot requests hands its team 100 exceptions, and a pilot team can work 100 by hand. Then the workflow succeeds. It improves to 95 percent, adoption takes volume to 10,000 requests, and the exception count is 500. The exception rate halved, volume grew tenfold, and the load on people grew fivefold. The growth is a byproduct of success, and nobody plans for the workload that comes with it.
In a pilot, exceptions feel manageable because the team absorbs them by hand. Someone curates the inputs, someone watches the runs, someone clears the odd case. None of that scales.
What follows happens often enough to be a composite, with details changed. A claims team at a different insurer stands up an intake workflow on a six-week timeline. The model reads incoming claims, checks them against the policy file, and clears the routine ones. The manual process routes problem claims through pend codes and desk procedures built up over years. The new workflow bypasses all of that and sends everything it cannot clear to a single escalation queue, worked by the senior adjusters. Documents can disagree with a policy file in a hundred ways, and every variety of mismatch now lands in that queue.
Within a week the queue is a backlog on the daily reports. Two months later it still is. Senior adjusters clear cases that need a rule rather than judgment, and the rest of the team drifts back to side spreadsheets. Every escalation points at a gap in the design, but closing gaps is nobody’s job.
Why Front-Loading AI Requirements Isn’t Enough
The obvious response is to front-load the design. A careful requirements review would have configured the procurement tolerance and wired in the pend codes, and most teams should do more of it. But some problems survive any amount of review.
Dave Snowden’s Cynefin framework separates complicated problems, which experts can analyze and settle in advance, from complex ones, where cause and effect only become clear in retrospect. Teams have to probe, sense, and respond. Most AI rollouts get treated as if the whole job were complicated, and much of it is. The rest turns complex when policies, systems, and assumptions collide in combinations nobody has seen. A requirements workshop cannot write a rule for a situation that has not happened yet.
Design what can be known in advance. For the rest, build a loop quick enough to learn in contact with live conditions. Model choice, prompt design, and orchestration each tune a part of the system, but what production delivers is the parts interacting, and no component test predicts that.
How a Learning Loop Closes the Readiness Gap
That job used to come with the schedule. On a long project, teams found the gaps early and revised the design before the full volume arrived. Shorter timelines ended that cycle; the need survived. Now the work has to be organized deliberately, as a learning loop. The workflow runs, the team captures what the design missed, somebody turns it into a standing rule, and human judgment concentrates on the cases the system cannot handle yet. Then the cycle repeats.
In Google’s 2025 DORA research, organizations with loosely coupled architectures and fast feedback loops reported AI productivity gains of 20 to 30 percent, while organizations tied to slow, legacy processes saw little or nothing with the same models available. DORA measures software delivery rather than the operation around it, but within its own domain, the gains tracked loop speed.
Loop speed is not a software idea. The Toyota Production System has treated standard work as a living document for seventy years. Run the work, capture what broke, update the standard, run it again. An AI workflow puts that practice on a tighter clock.
Go back to the claims team. Somebody takes ownership of the mismatch backlog and writes down how the senior adjusters resolve each variety. The write-ups become routing rules, mismatches within tolerance start clearing automatically, and within a few cycles the backlog is half its former size. The cases that still reach a senior adjuster are the ones that need one.
In The New Science of Management Decision (1960), Herbert Simon drew this distinction. Decisions that follow a rule are programmed. Decisions that need judgment are nonprogrammed. A learning loop keeps turning the second kind into the first.
Escalations feed only half the loop. Teams also need to sample what the system clears. A claim the model clears when it should have flagged it never enters a queue. It just gets paid. Claims people call that leakage, and no report surfaces it. Only a standing sample of the auto-cleared work, reviewed the same way as the escalations, catches it.
Over time, the boundary between human and machine trends in one direction. Whatever a team can specify, the agent takes over. People keep the calls that need judgment, the way the senior adjusters ended up with only the mismatches no rule covered, and they keep the sampling that checks the agent’s work.
What to Put in Place Before Launch
The learning loop needs routing, owners, thresholds, fallbacks, and a way to turn findings into standing rules. That takes five operational decisions, all of them yours to make before launch.
- Route the exceptions. Your people spent years building pend codes, queues, and desk procedures for running the work by hand. Map the cases the model cannot clear into that structure. Otherwise, all the varieties pile up together in one queue.
- Assign an escalation owner and a threshold. A vague threshold is how a hundred document mismatches end up on one adjuster’s desk.
- Choose how work keeps moving when the workflow stalls. A probabilistic system fails by drifting, not by crashing. Settle where cases go while quality is in question, who can stop the workflow entirely, and on what evidence. Teams that skip this end up back in side spreadsheets.
- Keep some calls human. Decide which calls never clear automatically, at any confidence level. Discovering that list in production costs far more than writing it now.
- Fund the rule-writing. The loop compounds only if someone writes what it surfaces back into the system. That writing is a job with a name on the org chart and hours in the week. Treated as a leftover, it does not happen, and every lesson gets relearned case by case.
Two more decisions belong to the COO, not to engineering. Name who forecasts the exception volume and staffs for it, and name who samples the work the system clears automatically.
The list is short on purpose. Before launch, nobody knows which exceptions will come, what the routing rules should say, or which cases will need human judgment. A plan that claims otherwise is describing the demo. The decisions set the structure. Running the loop fills in the content.
A workable shape for many teams is tiered, and operations leaders have run L1, L2, L3 models for decades. Routine, well-specified cases go to the AI. Ambiguous ones go to human specialists. The recurring ones go to people with the authority to change the design so they stop recurring.
Tier boundaries have always moved. Shift-left programs have pushed work down for twenty years, through knowledge articles and training cycles. The boundary now shifts weekly instead of quarterly. It shifts every time the model version changes whether anyone approved it or not, and sometimes it moves backward, because a regression means pulling work back from the AI. Somebody has to own that movement the way a program office used to own shift-left, except continuously.
A workflow can be designed, built, and demonstrated by a small group. It becomes an operation when the people who own the work trust it enough to run their normal caseload through it. Everything above, from the routing to the ownership to the loop itself, exists to clear that bar.
Build the Preparation Window Back In
Development got cheaper, and leaders took the gains, as they should have. What was easy to miss is that readiness got harder at the same time, and nobody schedules it anymore, because the six months it used to occupy are gone.
A century ago, the advantage did not go to the first factories with electric motors. Everyone could buy motors. It went to the owners who rebuilt the floor around the work. The models are for rent to everyone too, so build speed will not set anyone apart for long. An operation built to learn will, because it gets cheaper to run as it ages, which is the reverse of how most production systems behave.
The new speed pays out in saved time, and an hour saved is not an outcome. It is capacity that dissolves into the day unless somebody redeploys it, so put part of it into the work the tooling cannot do. Take the five decisions into your next launch, hand your COO the other two, give the tiers an owner, and stand up the learning loop before the volume arrives. Start with the workflow you are shipping this quarter. The schedule will not bring the window back; the loop is how you build it back in.
Who owns what your organization learns after launch?
Key Takeaways
- A development cycle compressed to weeks leaves little of the preparation time organizations relied on to absorb, staff, and integrate new capabilities.
- Once the constraint moves out of engineering and into the business operation, speeding up development only piles work in front of it.
- AI workflows cannot be proven correct in advance because they return different outputs depending on context, model version, and available data. Learning comes only from live volume.
- Exception volume grows as a byproduct of success. A workflow that improves from 90 to 95 percent accuracy while volume grows tenfold produces five times the workload for the people handling what the system cannot.
- A standing learning loop that converts recurring exceptions into rules is how you rebuild the preparation window that compressed timelines closed.
