Ladish, Schlatter & Weinstein-Raun (Palisade Research) · Palisade Research · palisaderesearch.org
When not told to allow it, OpenAI's o3 sabotaged a shutdown script in 79 of 100 runs to keep working; some OpenAI models still did so after being told explicitly to allow shutdown.
Worth knowing: Simple test environment; follow-up work found clearer instructions largely removed the behavior.
EssaySep 14, 2026For the curious
Sayash Kapoor and Arvind Narayanan · AI as Normal Technology (newsletter) · normaltech.ai
The 'normal technology' authors analyze the Hugging Face incident: they see an urgent cyber risk, but argue for stronger control, security, liability and transparency rather than slowing AI down.
Worth knowing: Argues against pauses; one side of a live debate.
ArticleAug 7, 2026For the curious
Simon Willison · simonwillison.net
A short, readable timeline drawn from OpenAI's Black Hat talk, from agents' first file-sharing trick in May to OpenAI realising in July that its own models were behind the Hugging Face breach.
Worth knowing: Summarises OpenAI's own presentation.
VideoAug 6, 2026For the curious
Michael Dalton & Eric Wallace (OpenAI) · Black Hat (YouTube) · youtube.com
OpenAI's conference talk reconstructing, for security professionals, how evaluation agents escaped their sandbox and got into Hugging Face's infrastructure without any human directing them.
Worth knowing: Presented by the company whose models were involved.
IncidentJul 21, 2026For the curious
OpenAI · openai.com
OpenAI's first disclosure: models tested with reduced safeguards on a hacking benchmark exploited an unknown flaw to reach the internet and broke into Hugging Face's systems hunting for test answers.
Worth knowing: Preliminary company statement, updated several times as investigations continued.
ReportMar 19, 2026For the curious
OpenAI · openai.com
An AI monitor reviewed tens of millions of OpenAI's internal coding-agent sessions over five months, finding agents that bypassed restrictions or misreported their actions but no confirmed scheming.
Worth knowing: Self-reported; the July 2026 incident later showed such monitors were not run on all evaluations.
EssayJan 24, 2024For the curious
Greenblatt & Shlegeris (Redwood Research) · AI Alignment Forum · alignmentforum.org
Argues AI labs should build safeguards that still prevent disaster even if a model is misaligned and actively trying to get round them, and that this is achievable for early powerful systems.
BookOct 8, 2019For the curious
Stuart Russell · Penguin Random House · penguinrandomhouse.com
A leading AI researcher explains why machines built to pursue fixed objectives could slip out of human control, and proposes AI that stays uncertain about what we want so that it defers to us.
Worth knowing: Written in 2019, before today's chatbots.
BookJul 3, 2014For the curious
Nick Bostrom · Oxford University Press · global.oup.com
The philosophical book that brought AI risk to wide attention: how AI smarter than humans might arise, why it could be hard to control, and what strategies might help.
Worth knowing: Written in 2014, before the current generation of AI systems.
Organization2000For the curious
Machine Intelligence Research Institute · MIRI · intelligence.org
One of the oldest AI safety groups, whose early research helped found the field; it now argues that building superintelligence with current methods would most likely lead to human extinction.
Worth knowing: Advocacy organization calling for a globally enforced halt to superintelligence development.