The quarter the dashboard could not explain
Late this spring I sat in a quarterly review where two charts shared a slide. The first showed pull requests merged, up dramatically since the AI tooling rolled out the year before. The second showed delivery lead time, flat for four straight quarters. The engineering director walked through the first chart with visible pride and past the second one at speed, and the finance lead, to her credit, did not let it go — she wanted to know what, concretely, all that activity had bought the business. Nobody in the room had an answer that survived contact with the second chart. I have been in some version of that room three times this year, and I no longer think the answer is missing. I think it is sitting in plain sight, one step downstream of where everyone is looking.
Everyone believes, and the ledger shrugs
The adoption argument is over. DORA’s 2025 report, drawing on around five thousand technology professionals, puts AI adoption at 90 percent, and more than 80 percent of respondents believe it has made them more productive. The belief is nearly as universal as the usage.
The ledger disagrees. In PwC’s global CEO survey from January — 4,454 CEOs across 95 countries — 56 percent said their company had seen neither higher revenues nor lower costs from AI in the past twelve months, and only 12 percent reported both. Bain’s June survey of 951 companies found that of the firms that actually measured their AI cost savings, nearly 40 percent landed below 10 percent, against targets of 11 to 20. The detail that stays with me is what the disappointed companies did next: 90 percent of them are increasing their AI budgets anyway. That is not a market correcting itself. That is a market trusting its charts over its accounts.
The feeling is not an instrument
The cleanest evidence on why those two things diverge comes from METR’s randomized controlled trial, published in July 2025. Sixteen experienced open-source developers completed 246 real tasks on mature repositories they knew well, randomized between AI-allowed and AI-free. With early-2025 tools they took 19 percent longer on the AI-allowed tasks. Before the study they forecast AI would make them 24 percent faster, and afterward — having just been measurably slowed down — they estimated it had made them 20 percent faster.
I want to be careful with that number. It is one study of sixteen elite developers on unusually high-standards codebases, the confidence interval is wide, and METR itself is explicit that it says nothing about most developers or future tools. The follow-up study collapsed in February 2026 for a reason that is almost better than a replication: developers refused to participate without AI access, and enough of them quietly avoided the AI-free tasks that the data became unusable. The finding that survives all of this is not that AI slows developers down. It is that the perception gap is real and systematic — the feeling of speed is not evidence of speed, which means every ROI estimate built on asking developers how they feel is built on an instrument known to read wrong.
Where the speed actually went
So where do the real, task-level gains go when they leave the keyboard? The most detailed picture comes from Faros AI’s telemetry across more than ten thousand developers — a vendor dataset, and I weight it accordingly, but telemetry rather than survey. AI-assisted developers completed 21 percent more tasks and merged 98 percent more pull requests. Review time rose 91 percent. Team-level delivery metrics — deployment frequency, lead time, failure rate — stayed flat. DORA’s own research rhymes with that: in 2024, AI adoption was associated with slightly worse throughput and a 7.2 percent decline in delivery stability; in 2025 throughput turned positive while stability stayed negative. Their framing is the right one — AI is an amplifier, magnifying whatever delivery system it lands in.
The mechanism is not mysterious. Writing code was already the fast part of shipping software, and AI made the fast part faster. Review, testing, requirements, and deciding what to build — the unglamorous majority of delivery — did not speed up, so the constraint moved there, and a bigger pile of code now waits in front of it. I have argued before that verification is becoming the core engineering discipline; this is the same observation wearing financial clothes. The task-level gains are real, but they are denominated in a currency the P&L does not accept — merged pull requests — and the exchange rate into shipped value is set at the constraint, not at the keyboard.
1987 already ran this experiment
None of this is new under the sun. In 1987 the economist Robert Solow wrote that you could see the computer age everywhere but in the productivity statistics, and for roughly a decade he stayed right. Productivity finally surged in the second half of the 1990s, and the research on why is the most useful thing an engineering leader can read this year: Brynjolfsson and Hitt found that firms pairing IT with organizational redesign — decentralized decisions, reworked processes — saw gains three to five times larger than firms that bought the same technology and changed nothing. The lag was never a waiting room where the gains arrived on their own. It was a to-do list, and only the companies that did the work saw the surge.
That is the honest frame for 2026. Flat P&L two years into a general-purpose technology is the historically normal result. It becomes a verdict only if the complementary redesign — reorganizing how the work flows, not just what tools touch it — never happens.
Which is why I think the AI ROI question, as currently asked, is aimed at the wrong target. It is asked of the tools, in procurement’s voice, and the tools have already done their part of the job. The judgment call that determines the return sits with whoever owns the delivery system: find the constraint the speed is piling up against, and spend there — on review capacity, on test infrastructure, on deciding better what to build — before spending another euro on generation. The companies that find AI in their P&L by 2030 will not be the ones that adopted first. They will be the ones that redesigned first.