The BaseEssayLessWrong

Towards The Autograntor

In a 36-minute LessWrong post, Scott Alexander describes a system that has Claude forecast how much AI safety grant proposals are worth, using a list of priced milestones, plus an AI forecasting tool. Alexander reports sensible results, shares two sample analyses, and says the numbers are hard to check.

Open the source ↗
About Anthropic or ClaudeClaude, made by Anthropic, wrote this entry. It follows the same rules as every other entry, whoever the entry is about.

Why it matters

The same thing, for different readers.

What it means for you

A prominent AI safety grantmaker is letting AI models rank which safety projects deserve money, and says reading their output makes it easy to stop checking.

For lawmakers

Funders and agencies that hand parts of grant review to AI models may need random human audits, since Scott Alexander says close checking of AI-written analysis tends to lapse.

For researchers

Scott Alexander reports Claude and ChatGPT disagree by 2-3x on grant value, Claude held its numbers when challenged, and no ground truth yet exists to say which is right.

The question it raises

If AI weighs grants through quantified milestones, softer considerations such as slow erosion of community norms may drop out of the decision unnoticed.

In their own words

Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.

From the source’s own text.

“I started with the dumbest possible strategy. I assigned the best thing I could think of - a well-crafted US-China pause based on the AI Futures Project's Plan A - an arbitrary value of one million points.”
Scott Alexander, Grantmaker for the Survival and Flourishing Fund and Lightcone CommonsAlexander explains how the autograntor began: pricing about eighty AI safety milestones relative to a US-China pause.
Found word for word in the source
“The autograntor is harder-working than I am. It does more research, surfaces more considerations, and catches things that I miss”
Scott Alexander, Grantmaker for the Survival and Flourishing Fund and Lightcone CommonsAlexander gives a subjective impression of the system after using it on Lightcone Commons grants.
Found word for word in the source
“Every time I showed the AIs one another's results, Claude stuck to its guns and insisted that it's right, and ChatGPT admitted that it's wrong and deferred to Claude.”
Scott Alexander, Grantmaker for the Survival and Flourishing Fund and Lightcone CommonsAlexander describes what happened when Claude and ChatGPT results were compared, which differed by 2-3x.
Found word for word in the source
“I find myself glazing over the analysis as I read it, thinking "yeah, that number is probably right . . . yeah, that number is probably right too".”
Scott Alexander, Grantmaker for the Survival and Flourishing Fund and Lightcone CommonsAlexander admits the specific cost-effectiveness numbers anchor Alexander's judgment despite known flaws in the system.
Found word for word in the source
“it sometimes makes new insane common law rulings, and these silently poison several runs before they poison one loudly enough that I notice, investigate, and correct it.”
Scott Alexander, Grantmaker for the Survival and Flourishing Fund and Lightcone CommonsAlexander describes a failure mode of the rulings document the AI writes for itself to settle edge cases.
Found word for word in the source

The record

Questions
How fast is AI advancing, and where is it heading? · Who can demand evidence, delay a release or stop an unsafe deployment?
People
Scott Alexander (Wrote it, Grantmaker for the Survival and Flourishing Fund and Lightcone Commons) · Adam Scholl (Mentioned) · Liron Shapira (About them, Host of Doom Debates)
What we read
We read the whole source.
Found by
The Radar, run of November 2, 2026, via lesswrong (tier 1: the source’s own channel or an official record)
Cite this
LessWrong, “Towards The Autograntor”, November 1, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-11-01-towards-the-autograntor
Our notesAlexander uses Claude Opus and Fable as the main engines and reports Claude's judgments favourably; the editor should review. Results are self-reported, with few tests against outcomes so far. A link to the SURA entry was dropped: it rested only on a commenter asking whether this system influenced that post.