The software did not eat the workflow

Ask a developer whether AI made him faster this quarter and he will say yes — about twenty percent. Put a stopwatch on him instead: randomised, controlled, on his own mature codebase. He is nineteen percent slower. The distance between those two numbers is forty points, and it was measured, not argued.

Now notice something about every AI business case you have ever been shown. It was built on the first number.

Three years and several hundred billion dollars into enterprise AI, only 21% of the S&P 500 can point to a measurable benefit; 42% of firms abandoned most of their initiatives last year; Goldman still cannot find a relationship between AI adoption and productivity in the aggregate; and the bond market now charges roughly a 30 basis point spread penalty for being an AI "adopter" rather than an "enabler" — the story, priced, without the cash flows. Meanwhile the models get measurably better every quarter and capex runs toward $675 billion. Both facts are real. That is the interesting part.

The models are not failing. The unit of deployment is wrong. The industry bought a translation layer and deployed it as an employee. This piece is that sentence, with evidence — and with the three mechanisms that keep the spend from ever reaching the accounts.

The gap. The controlled trials are not contradictory; they are a map. On bounded, self-contained work AI wins big and repeatedly: 56% faster on a scoped coding task, 26% more pull requests across five thousand developers, novice support agents up 34%. At the expert, embedded end — real issues, mature systems — the sign flips, while still feeling like a speedup. The felt number is the industry's measurement instrument, and the one time it met a stopwatch it missed by forty points. Every "hours saved" survey on every board deck sits on the wrong side of that gap.

The leak. Watch what people actually do with the tools: of three thousand observed work tasks, the top ten account for a quarter of all usage — drafting, debugging, summarising, reformatting. The commonalities. Acemoglu's arithmetic put the profitably automatable share of US work at 4.6% of tasks, and three years of aggregate data now look more like his 0.7%-per-decade world than the seven-trillion-dollar decks. Worse, part of each saving is spent downstream: AI output polished enough to forward and hollow enough to redo costs the receiving colleague about two hours an incident, and employees report that 37% of the time AI saves them is clawed back correcting its output. The gain is real at the desk and evaporates in the aggregate. Solow saw this exact shape in 1987.

The wrapper. The AI employees are allowed to touch is not the AI in the demo. The sanctioned deployment is a wrapper — older approved models, no API keys, no code execution, every upload scanned — each control defensible, the stack structurally weaker than the free consumer product. Procurement approves in quarters while the frontier turns over in weeks, so the firewall does not just block data leaving; it blocks capability arriving. Employees resolved this the way employees always do: roughly 47% of workplace AI use now runs through personal accounts, with senior leaders twice as likely as their own teams to use unapproved tools. What the data proves is narrow but damning — blocking harder recovers nothing.

Three mechanisms, and note what they share: none is a model deficiency. They are properties of how the thing was deployed — as a seat, per person, measured by feel, behind a wall. That is the copilot pattern, and it cannot compound.

The standard rebuttal is that copilots were merely act one, and agents — whole workstreams handed over end to end — are the payoff. Credit where due: on short desktop tasks frontier agents now clear about 85%, above the measured human baseline. The blanket claim that agents fail 70% of the time is stale.

But two structural facts survive every model release. First, a language model has no ground truth: it produces the most probable continuation, and OpenAI's own researchers proved in 2025 that a floor rate of confident falsehood is mathematically guaranteed by how these systems are trained and scored. The manufacturer has published a proof about the limits of the manufacturer's product. Second, errors compound and text does not fail loudly. A workflow is a chain of dependent steps, and the arithmetic is merciless.

Unlike a crashed program, a wrong paragraph does not throw an exception. It reads fine. The failure travels downstream wearing a suit.

To be precise, because the sophisticated pushback is real: agents can call deterministic tools, retry, verify against tests — and every one of those fixes moves the reliability out of the model and into scaffolding. The constraint is narrower and harder than "agents don't work." The model cannot be the source of truth, and it cannot be the control plane for critical state. Wrap it in enough rails to make the chain safe and you have not refuted the thesis. You have built it.

The audited successes share one architecture, and it is boring on purpose. Nobody hands the model a workflow; they give it a seat in one. An invoice or a loan file arrives; the model extracts the fields from the mess; code validates them against the system of record; a rules engine computes the number; the model drafts the explanation; a human sees only the exceptions. Every number the customer sees came from code. Every sentence came from the model. Neither did the other's job.

Language at the edges, determinism at the core.

Notice what this does to the three mechanisms. The gap closes because the KPI is cycle time on a process, not self-reported hours. The leak closes because wrongness is caught by validation rails, not by a colleague two desks downstream. The exponent never fires because the chain of dependent steps runs through code, and the model sits at one or two joints instead of owning it. Goldman's aggregate found nothing economy-wide but a roughly 30% median gain in exactly these narrow, instrumented cases. The signal is not missing. It is narrow — and the narrowness is the design specification, not the disappointment.

The precedent is a century old. Electricity took forty years to reach the productivity statistics, and the gains arrived only when factories were rebuilt around the motor rather than bolting it where the steam shaft had been. We are in the swap-in phase of AI — a chatbot where the steam shaft was — measuring swap-in returns and calling the technology overrated.

Change the instrument. Cycle time on an end-to-end process, error rates at the point of downstream consumption, cost per completed workflow. Never seats, never adoption, never self-reported hours — the controlled evidence says those run forty points hot, with total sincerity.

Change the unit. Deploy the model at the joints, not over the chain: extraction, classification, drafting — with code and systems of record supplying every fact that must be true. The integration work that makes this possible is the underpriced line in every AI budget, because it does not demo.

Change the trade. A hobbled tool plus no training produced the shadow economy. The reverse trade — frontier capability inside the perimeter, plus training the one skill that pays, knowing where the model's competence ends in your own job — costs a fraction of the licence bill and is worth twenty quality points either way, because the same experiment that found +40% inside the frontier found experts nineteen points worse outside it, anchoring on fluent wrong answers.

The gap. The leak. The wrapper. And underneath all three, the exponent — the arithmetic that says a probability distribution must not own a chain whose value depends on being exact.

The models are extraordinary, and the audited corporate return is a minority outcome, and both are true because organisations bought a layer and staffed it like an employee. Employees figured out the truth on their personal accounts: it is brilliant at the small stuff. The firms with returns a CFO can sign took the hint — language at the edges, determinism at the core, and nobody asking a probability distribution to sign off on the numbers.

The software will get its bite of the workflow. It will take it the way software always has: through re-architecture, slowly, in the back office, long after the keynote. The demo was never the product. The plumbing is the product.