Joel Becker

“Surprisingly, we find that allowing AI actually increases completion time by 19%—AI tooling slowed developers down.”

“Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%.”

“This slowdown also contradicts predictions from experts in economics (39% shorter) and ML (38% shorter).”

“Although the influence of experimental artifacts cannot be entirely ruled out, the robustness of the slowdown effect across our analyses suggests it is unlikely to primarily be a function of our experimental design.”

“That said, many of the factors we find evidence for contributing to slowdown are specific to the setting we study—these results do not imply that current AI systems are not useful in many realistic, economically relevant settings.”