The BaseEssayLessWrong
"About a quarter of them contained bugs": Gemini 4's ML R&D benchmark
On LessWrong, Simon Mendelsohn examines Google DeepMind's Gemini 4 Argon safety report, which rules out its AI-research-automation threshold using an internal benchmark that the report's own audit says has about a quarter of tasks unpassable. Mendelsohn agrees with the verdict but asks Google to disclose the numbers.
Why it matters
The same thing, for different readers.
What it means for you
The tests used to say whether a new AI could automate AI research were partly broken, so a reassuring result says less than it appears to.
For lawmakers
Safety frameworks that rely on pass-rate thresholds need disclosed denominators and audited benchmarks; the author argues regulation should require more transparency of this kind.
For researchers
Unpassable tasks only lower a capability score, so bugs bias rule-out tests toward false reassurance; the post requests scored subset size, bug counts and timeouts.
The question it raises
If a refusal triggered by evaluation awareness looks the same as failure, tests cannot separate what a model won't do from what it can't.
In their own words
Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.
From the source’s own text.
“During this evaluation cycle, we audited the subset of original challenges that remain in the current suite. About a quarter of them contained bugs that prevent any submission from passing (e.g. crashing verifiers, incorrect ground truth), and several more were graded against details not stated in the task.”
“Our conclusion that Gemini 4 is below the threshold is robust to these issues: Gemini 4 would remain below the threshold even if every incomplete trajectory had succeeded.”
“An unpassable task only lowers the score, never raises it. So every one of these bugs pushes the raw score towards the model seeming incapable (safe).”
“I’m not reassured that Google will actually be able to tell if the next model passes thresholds.”
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A reading of Google DeepMind's Gemini 4 Argon safety report and public summaries of it, focused on the internal ML R&D benchmark (GRB) used to rule out the automation threshold.
- What it found
- Per the report as quoted, about a quarter of audited original tasks had bugs preventing any pass; many runs timed out; of completed runs the model solved roughly half, and Google says the conclusion holds even counting bugged tasks or unfinished runs as passes.
- What it does not show
- The post cannot see the benchmark and relies on the report's text; the size of the scored subset, the number of bugged tasks and the denominator of the 90% rule are not disclosed.
- Status
- A report
The record
- Questions
- What happens when AI helps build more capable AI? · Are AI companies putting stronger safeguards into practice?
- People
- Simon Mendelsohn (Wrote it)
- What we read
- We read the whole source.
- Found by
- The Radar, run of October 17, 2026, via
lesswrong(tier 1: the source’s own channel or an official record) - Cite this
- LessWrong, “"About a quarter of them contained bugs": Gemini 4's ML R&D benchmark”, October 16, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-10-16-gemini-4-ml-rd-benchmark-bugs