The BaseResearchLessWrong
GPT-6.2 Astra scores below chance on TASTE; GPT-6 Astra and Fable 5.2 near chance
In a LessWrong post, Sarah Wang reports running the TASTE benchmark, which asks models to pick the better of two AI safety research proposals, on three newer models. Wang finds GPT-6.2 Astra below chance and the other two near chance.
Why it matters
The same thing, for different readers.
For lawmakers
Claims that AI is already speeding up AI research are hard to check; this result shows one proxy for research judgment where newer models do not do better than chance.
For researchers
Wang's run adds three newer models to TASTE and finds none above chance, which questions whether the benchmark picks up recent AI R&D uplift or only its own quirks.
In their own words
Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.
From the source’s own text.
“Overall, none of the models tested score significantly above chance, and GPT-6.2 Astra scores significantly below it.”
“Any uplift in AI R&D does not seem to show up in TASTE in its current form.”
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A single author ran the TASTE benchmark (92 pairs of synthetic safety research proposals with gold labels set by researchers) on GPT-6.2 Astra, GPT-6 Astra and Fable 5.2, in two of the paper's four variations.
- What it found
- Astra 6.2 agreed with gold labels 35% of the time, Astra 6 41% and Fable 5.2 53%. With 92 pairs, the post puts the 95% interval for random guessing at 40-60%.
- What it does not show
- Small benchmark (92 pairs), only safety proposals, only two of four variations run, and a separate post finds that choosing the proposal with fewer mech-interp terms beats expert agreement.
- Status
- A report
The record
- Questions
- What happens when AI helps build more capable AI? · How fast is AI advancing, and where is it heading?
- People
- Sarah Wang (Wrote it)
- What we read
- We read the whole source.
- Found by
- The Radar, run of November 19, 2026, via
lesswrong(tier 1: the source’s own channel or an official record) - Cite this
- LessWrong, “GPT-6.2 Astra scores below chance on TASTE; GPT-6 Astra and Fable 5.2 near chance”, November 18, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-11-18-astra-below-chance-taste