AI adoption: developers 19% slower, believing they were 20% faster

Most engineering leaders I talk to describe the same eighteen months. Throughput went up. So did the number of changes that came back. Somewhere in that stretch a release broke something that had been stable for a year, and nobody could say whether the tools had helped or hurt.

I don't think that's a coincidence, and the published evidence says it isn't. DORA's 2025 survey of nearly 5,000 technology professionals, run between 13 June and 21 July 2025, found 90% use AI at work and more than 80% believe it made them more productive. The same survey found adoption correlates positively with throughput and product performance, and negatively with delivery stability.

Both things are true at once, and the rest of this piece is about why. The short version is that generation got cheap and checking didn't, so the constraint moved to the part of the process nobody was measuring.

Most of the AI conversation treats the model as the unit of progress. A better model arrives, capability jumps, the deployment problem shrinks. I don't think the published evidence supports that reading.

The position the people who build these systems have settled on is narrower. What ships is a system, and the model is one part of it. Anthropic's engineering writing treats context as "a critical but finite resource" to be curated rather than maximised. DORA's research finds AI's impact decided by the environment it sits inside. The multi-agent failure literature puts almost every failure in system design rather than model capability.

If that's right, then the work that decides whether a deployment succeeds is engineering work, and most of it happens before and after the model is called. I checked that claim against the published research and the code-quality data, and it holds up better than I expected.

The most useful single study I've found is METR's randomised controlled trial, published in July 2025. Sixteen experienced open-source developers worked 246 real tasks in their own repositories, which averaged over a million lines of code and 22,000 stars. Each task was randomly assigned to allow or forbid AI, typically Cursor Pro with Claude 3.5 or 3.7.

Before the study, the developers forecast a 24% speedup. After the work, they estimated they had been 20% faster. They were 19% slower.

That result is the one I keep coming back to, because it isn't a survey of opinions. DORA's trust finding sits alongside it: 30% of respondents report little or no trust in AI-generated code, in the same sample where more than 80% report a productivity gain. Both numbers can be true. People feel faster and trust the output less, and neither feeling is a measurement.

So a deployment needs a before-and-after on real work. Cycle time on comparable tickets, review rounds per pull request, escaped defects per hundred changes. The perception gap is 39 points wide in METR's data, which is wider than most deployment decisions can absorb.

DORA's central 2025 finding is that AI behaves as an amplifier. It magnifies the strengths of a high-performing organisation and the dysfunctions of a struggling one, which is why the same tooling helps one team and hurts the next.

The mechanism is not mysterious. AI raises the volume of change a team produces. When review, testing and rollback capacity don't rise with it, the extra change shows up as more failed deployments and slower recovery. Teams on loosely coupled architectures with fast feedback loops see gains. Teams on tightly coupled systems with slow processes see little or nothing.

The code-quality data puts a number on it. Refactoring fell 70% against 2022 and long-term legacy maintenance fell 74%, across the 623 million code changes GitClear analysed from 2023 to 2026. Block duplication rose 81% and error-masking 47%. Its 2025 report had already found the share of changed lines that were refactored fall from 25% in 2021 to under 10% in 2024, while copy-pasted lines rose from 8.3% to 12.3%.

GitClear sells code analytics, so I read those numbers as a vendor's own telemetry rather than a controlled study. The direction matches DORA's stability finding, which is independent. Throughput gains arrive first and the maintainability cost lands later, on whoever owns the codebase in eighteen months.

DORA's seven capabilities are the practical answer, and they're unglamorous. A clear and communicated AI stance, healthy data ecosystems, AI-accessible internal data, strong version control, working in small batches, user-centric focus, and a quality internal platform. None of them is a model.

Context is the interface to the model, and that's where the engineering effort goes first. Anthropic's September 2025 write-up on context engineering frames the problem as curating "the smallest set of high-signal tokens" that maximises the chance of the outcome you want. The research behind that instinct is Liu and colleagues' 2024 paper in Transactions of the ACL. They found model performance highest when relevant information sits at the beginning or end of a long input. It degrades when the information sits in the middle, even for models built for long context.

Context files ought to pay off, then. Two 2026 studies tested them and got different answers.

Lulla and colleagues tested AGENTS.md files across 10 repositories and 124 pull requests. Median wall-clock runtime fell 28.64%, from 98.57 to 70.34 seconds, and output tokens fell 16.58%, from 2,925 to 2,440, with no loss of task completion. More than 60,000 repositories now carry such a file.

A separate study on arXiv, 2602.11988, tested three conditions on a benchmark built from real repositories and pull requests: no context file, an LLM-generated one, and a developer-written one. LLM-generated context files reduced task success by up to 3% against no file at all, and raised inference cost by more than 20%. Developer-written files beat the baseline by around 4%, at higher token cost and more reasoning steps.

Both results can hold. A hand-written context file makes an agent navigate faster and cheaper. A generated one adds tokens without adding signal, and can talk the agent into the wrong approach. Write the file yourself, keep it short, and measure correctness separately from cost, because the two move in different directions.

Cemri and colleagues at Berkeley analysed seven popular multi-agent frameworks across more than 200 tasks with six expert annotators for MAST, the Multi-Agent System Failure Taxonomy, published at NeurIPS 2025. They found failure rates between 41% and 86.7%. The taxonomy has 14 failure modes in three categories: specification issues, inter-agent misalignment, and task verification. The most common were step repetition at 15.7%, reasoning-action mismatch at 13.2%, and agents unaware of when to terminate at 12.4%.

Fixes are structural. Giving one framework's coordinating agent final say over termination lifted task success 9.4%, which is a design change, not a prompt change. The failure categories are also nearly uncorrelated, with inter-category correlations of 0.17 to 0.32. Fixing one mode doesn't fix another, so you diagnose before you re-architect.

A multi-agent system is an organisational design with the same problems as a team. Unclear briefs, duplicated work, nobody owning the stop condition. Adding agents to a broken workflow multiplies the failure, and the taxonomy is the checklist.

The findings converge on one constraint. Generation got cheap; checking didn't.

DORA finds instability rising with adoption, GitClear finds refactoring and legacy maintenance collapsing, and MAST puts task verification among its three failure categories. METR's developers were slower on work they already understood, which is what happens when review and debugging absorb the time generation saved.

The work this leaves is mostly unautomated: intent, specification, acceptance criteria, review, verification, quality gates, and feedback loops after release. The scarce skill in an AI-heavy delivery process is knowing what correct looks like before the work starts, and being able to prove it after.

We run two rules on our own delivery for this reason. No agent submits anything it produced: it drafts, a person reviews and submits, and the agent has no authority over the final state. Every push to a client site runs the UI and responsiveness checks before a person looks at it. Review then starts from a known baseline rather than from whatever the agent believed. Neither rule is clever. Both exist because the alternative is a person reviewing at the speed the agent produced, which is not review.

Anthropic ran its own trial, published in January 2026 as arXiv 2601.20245, with 52 mostly junior engineers learning an unfamiliar Python library. The group with AI access finished about two minutes faster, which wasn't statistically significant. On a comprehension quiz covering concepts they'd used minutes earlier, they scored 50% against 67% for the group that coded by hand. The largest gap was in debugging.

Inside the treatment group the split was wider still. Participants who used AI to ask conceptual questions and build understanding scored 65% or higher. Those who delegated code generation scored below 40%.

That's a design choice available to any team, and it's the one I'd spend the most time on. AI as an explanation engine builds the judgement you need to supervise it. AI as a code vending machine removes the practice that judgement comes from, and the cost lands when someone has to debug work nobody understood in the first place.

Portability used to be an aspiration. It's now an architectural option.

Anthropic donated the Model Context Protocol to the Linux Foundation's new Agentic AI Foundation on 9 December 2025. The foundation is a directed fund co-founded by Anthropic, Block and OpenAI, with AWS, Bloomberg, Cloudflare, Google and Microsoft as platinum supporters. Its founding projects are MCP, Block's goose and OpenAI's AGENTS.md. Google handed the Agent2Agent protocol to the Linux Foundation in June 2025, and Anthropic donated Agent Skills to the AAIF in October 2026.

The protocols that connect agents to tools, to each other and to their instruction files now sit under neutral governance. That doesn't make switching free, but it makes the exit real, which changes the procurement conversation. A deployment that depends on one vendor's proprietary connector is now a choice.

One objection to all of this is cost. If tokens are expensive, small models and tight context win on economics, and the engineering layer is a cost-reduction exercise.

Epoch AI's analysis of inference pricing puts the fall at roughly 13 times a year for constant capability over five years. For comparison, US residential electricity fell 1.05 times a year from 1892 to 1973, lithium-ion batteries 1.16, compute 1.51 and DNA sequencing 1.84. GPT-3 cost $60 per million tokens at an MMLU score of 43.9. Llama 2-7B reached 45.3 at $0.20, a compound decline of about 31 times a year.

If price keeps falling that fast, the durable constraints are latency, context quality and verification capacity, not the invoice. That strengthens the case for the engineering layer. You don't build a control plane because tokens are cheap this quarter. You build it because the thing that fails is the workflow, and the workflow doesn't get cheaper on its own.

If I were starting this again, I'd do two things before anything else.

The first is to measure. Take two of the metrics above and run a before-and-after for a month. METR's 39-point perception gap and DORA's trust finding both say self-report is unusable, so the number has to come from the tracker rather than from the team's sense of itself.

The second is to raise review and test capacity before raising change volume. DORA's stability finding and GitClear's maintainability data describe the same cost arriving twice, and the second arrival is the expensive one.

1. Write the context file by hand and keep it short. Generated context files cost more and can lower success. Hand-written ones cut runtime by a quarter.
2. Give every agent a stop condition. The most common multi-agent failure is not knowing when to terminate, and the fix lifted success 9.4%.
3. Diagnose before you re-architect. MAST's failure categories barely correlate, so a fix aimed at the wrong mode buys nothing.
4. Use AI to build understanding, not to replace it. The comprehension gap is 17 points overall and much wider between the two ways of using it.
5. Assume you'll switch vendors. MCP, AGENTS.md and Agent Skills sit under neutral governance now. Design as if the exit is real, because it is.
6. Watch the invoice, but not first. Inference prices fall about 13 times a year for constant capability. Latency, context quality and verification capacity are the constraints that stay.

The engineering layer has a shape, and the evidence puts it in order: context, boundaries, verification and orchestration. Most teams I meet are working on the first of those and calling it a strategy.

Naming the layers is what makes it possible to ask where a deployment is thin. For most teams the answer is verification, because it is the one that costs money before it returns any, and the one that decides whether the rest of the investment holds.

The tools are genuinely good and getting better. What they don't do is fix a review process that was already the bottleneck, and that's where most of the value is sitting.