The BaseResearchLessWrong
Measuring agent capabilities on open-ended AI R&D tasks
According to the excerpt, Alex Jackson reports work by Arrow Research measuring AI agents on four open-ended AI R&D tasks. The post says frontier scores have a doubling time of around one year since 2024, slower on conceptual tasks than engineering-heavy ones.
Why it matters
The same thing, for different readers.
For researchers
Arrow Research's Alex Jackson reports agents improve faster on engineering-heavy than conceptual research tasks, though the task set is small and LLM-graded.
The question it raises
Alex Jackson's results suggest AI doing AI research advances unevenly, with conceptual work lagging engineering.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- Four difficult open-ended AI R&D tasks scored by automated checks and LLM judges using expert-written rubrics.
- What it found
- Overall doubling time just under one year based on an exponential fit; between 7 and 16.8 months median doubling time across task types, slower for conceptual tasks.
- What it does not show
- The author states a small set of tasks and heavy reliance on LLM grading.
- Status
- A report
The record
- Questions
- What happens when AI helps build more capable AI? · How fast is AI advancing, and where is it heading?
- People
- Alex Jackson (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of October 27, 2026, via
lesswrong(tier 1: the source’s own channel or an official record) - Cite this
- LessWrong, “Measuring agent capabilities on open-ended AI R&D tasks”, October 27, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-10-27-agent-capabilities-open-ended-ai-rd
Our notesLight lane: excerpt only. Excerpt names a best model; not repeated here as it was cut off.