Nate Rush
Model Evaluation & Threat Research (METR)
Titles as the sources gave them. In 1 entry, with 0 checked quotations.
In the Base
Every entry they appear in.
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
METR researchers report a randomized trial in which 16 experienced open-source developers completed 246 real tasks in mature repositories, each task randomly allowing or barring early-2025 AI tools. Tasks took 19% longer with AI allowed, although developers had expected a 24% saving. The authors caution against generalizing beyond this setting.