# A METR randomised trial of 16 experienced open-source developers on 246 real issues found they took 19% longer with early-2025 AI tools, while believing they had been 20% faster

Page: https://flyvolo.ai/en/changes/ev-20250710-junior-software-developer-2
Date: 2025-07-10 · Stage: Constraint
Can this record move a task judgement? yes
Occupation: https://flyvolo.ai/en/careers/experienced-software-engineer

## Scope

Experienced maintainers on large, mature repositories they know well (about 5 years each); tools were mainly Cursor Pro with Claude 3.5/3.7 Sonnet. The authors explicitly do not claim the result generalises to most developers or to greenfield or junior work.

## Source

- METR — study report — https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ (primary source)

## What this means

A constraint record on the capability layer with a measured result: in a randomised trial, experienced maintainers took 19% longer with early-2025 tools on real issues in repositories they knew, while believing they were 20% faster. For the linked tasks it caps how far 'routine code is generated' may be read as 'the work got faster', and the perception gap shows developers' own sense of speed is not a reliable measure of it.

## What it does not show yet

The authors say so themselves: 16 developers on mature repositories they had worked in for about five years, using tools that have since changed. It does not show junior developers get slower — they lack the repository knowledge that made the tools less useful here — and it does not measure code quality, learning, or greenfield work. A slowdown in this setting is not a slowdown in yours.

## How to verify it yourself

For one week, before starting each ticket write down your time estimate with and without the assistant, then record the actual time; METR's finding was the gap between those columns, and a personal log of ten tickets shows whether you have the same gap.


More: https://flyvolo.ai/llms.txt