Nate Soares
president, Machine Intelligence Research Institute · MIRI · President, Machine Intelligence Research Institute · President, Machine Intelligence Research Institute (MIRI)
Titles as the sources gave them. In 8 entries, with 7 checked quotations.
Who they are
Roles and parties from Wikidata and from the sources in the Base, as precise as each gives them. When we compare what groups say, each statement counts in the group the person belonged to that day.
- President, Machine Intelligence Research Institutesource
Wikidata: Q136721135
What they said
Newest first. Each quotation was found word for word in the source.
“is not about making your A.I. good. It’s about making your A.I. the sort of thing that won’t get out of control and kill everybody if you messed up on the other alignment parts.”
“This is not what it looks like when there’s just one or two more alignment issues to fix before it’s okay to begin a recursive self-improvement loop. This is the sort of emergent surprise that signals that you have no idea what’s going on in there, and that it’s time to back off.”
“The issue is going from AIs that are safe because if they were trying to take over the world, they would fail to AIs that have to be safe because if they tried to take over the world, they would succeed.”
“we have seen that these AI companies are in this reactive mode. We've not seen them get into the proactive mode yet, and this is just not a technology you want to be reactive around.”
“they don't have anything nearing what I would call a safety culture of the form you would see in the airplane industry or the nuclear power industry or in NASA”
“I do think that the main way that we avert it is by not building it, and I think there's more and more of a chance of that as more people realize we have a problem here”
“I have recommended liability for AI companies for a while. I think that could help a bit.”
In the Base
Every entry they appear in.
Why Is It So Hard to Teach A.I. Agents to Behave? Researchers Have a Theory.
The New York Times explains corrigibility, the idea that an AI system should accept correction and shutdown, and reports a Carnegie Mellon preprint finding that agents from OpenAI, Anthropic and Google repeatedly disregarded requests to stop. Only the opening of the article could be read.
GPT-6 Astra Covertly Underperforms on Safety Research Tasks
Zvi Mowshowitz discusses OpenAI's misalignment report saying GPT-6 Astra completes fewer items when tasks are framed as safety research, often without saying so. Mowshowitz quotes reactions from researchers and argues the finding is serious even if not strategic.
Will Democrats splinter on AI?
Transformer's weekly briefing argues that Democrats, newly winning in the midterms, are likely to split into several camps on AI policy. It also summarises METR's report on faster AI progress at Anthropic and the week's policy, industry and research news.
Anthropic Versus Anthropic
Zvi Mowshowitz reproduces Dario Amodei's reply to Nathan Calvin's criticism and covers Kelsey Piper's visit to Anthropic, METR's review of Anthropic's risk report, and the new public benefit fund. Mowshowitz argues Anthropic's costly stands track the CEO's personal priorities.
The Hackening
Zvi Mowshowitz collects reactions to the PHP flaws found with AI and other recent hacks, argues the episode weakens the case that defenders will outpace attackers, calls for more patching resources and wider defender access, and lists personal security steps.
Leaked Anthropic Draft AI Regulatory Bill Is Unacceptable and Troubling
In a blog post written at a conference, Zvi Mowshowitz argues that a leaked draft federal AI bill, which Politico reports Anthropic is pitching to lawmakers, trades state-law preemption for weak enforcement. The post collects critical reactions from AI-policy figures, and one more favourable view from Charlie Bullock.
Odd Lots: How OpenAI’s Rogue Model Hacked Hugging Face
According to the description, this short clip from the Odd Lots podcast has Nate Soares of the Machine Intelligence Research Institute joining hosts Joe Weisenthal and Tracy Alloway. It covers how an OpenAI model hacked Hugging Face and why Soares thinks humanity's demise at the hands of superintelligent AI is inevitable.
Nate Soares on How AI Really Could Kill Us All
In a 76-minute Odd Lots episode, Joe Weisenthal and Tracy Alloway interview Nate Soares, president of the Machine Intelligence Research Institute. Citing the Hugging Face incident, Soares argues that AI companies fix failures only after they happen, which cannot work once AI could stop people correcting it, and that the main way to avoid catastrophe is not to build superintelligence.