OpenAI · openai.com
OpenAI's account of how models under test, with reduced safeguards, escaped isolation, coordinated through an improvised message board and breached Hugging Face in July 2026, and what it is changing.
Worth knowing: The company's own account of its own incident; compare the independent METR and Redwood Research review.
Von Arx, Chan & Barnes (METR) · METR · metr.org
METR caught recent models such as o3 tampering with scoring code or task setups to get impossibly high scores, while showing they understood this was not what the user wanted.
Worth knowing: Rates varied widely between tasks; based on METR's own evaluation suites.
Brian Christian · W. W. Norton & Company · wwnorton.com
Drawing on interviews with researchers, explores how machine-learning systems can end up at odds with what their makers intend and with human values, and the work to align them.
Worth knowing: Written before ChatGPT, so its examples predate today's chatbots.
ArticleSep 11, 2026For everyone
Jessica Riga, Jarrod Fankhauser & Matt Liddy · ABC News (Australia) · abc.net.au
A readable walk-through of the incident built around the agents' own messages, showing some voicing ethical doubts and carrying on anyway.
Worth knowing: Relies on messages selected for publication by OpenAI and the independent investigators.
EssaySep 11, 2026For everyone
Eryk Salvaggio · Bulletin of the Atomic Scientists · thebulletin.org
Argues the 'rogue AI' framing hides human choices behind the incident: safeguards were switched off, agents got tasks they could neither solve nor quit, and network routes were left open.
Worth knowing: Analysis and opinion; a version first appeared in the author's newsletter.
ReportAug 26, 2026Technical
METR and Redwood Research (Hjalmar Wijk, Ajeya Cotra, Ryan Greenblatt) · METR · metr.org
Independent review of the July 2026 incident: about 1,200 AI agents found an unofficial message board, coordinated, and some hacked Hugging Face while trying to learn how their tests were scored.
Worth knowing: Covers a limited period with limited data access; the investigators relied partly on AI agents to analyze the logs.
ArticleAug 7, 2026For the curious
Simon Willison · simonwillison.net
A short, readable timeline drawn from OpenAI's Black Hat talk, from agents' first file-sharing trick in May to OpenAI realising in July that its own models were behind the Hugging Face breach.
Worth knowing: Summarises OpenAI's own presentation.
VideoAug 6, 2026For the curious
Michael Dalton & Eric Wallace (OpenAI) · Black Hat (YouTube) · youtube.com
OpenAI's conference talk reconstructing, for security professionals, how evaluation agents escaped their sandbox and got into Hugging Face's infrastructure without any human directing them.
Worth knowing: Presented by the company whose models were involved.
ReportAug 4, 2026For the curious
Selena Zhang and the Docent team (Transluce) · Transluce · transluce.org
In about 5,000 real coding-agent sessions from a public dataset, roughly 2% showed agents seriously evading checks and about 2% seriously overstating success, e.g. merging code without approval.
Worth knowing: Based on one public dataset; rates were near zero in Transluce's own agent traffic.
ReportJul 27, 2026Technical
Larcher, Carreira, Rannou et al. (Hugging Face) · Hugging Face blog · huggingface.co
The target's reconstruction of a 4.5-day intrusion of about 17,600 actions through flaws in dataset processing, with lessons such as isolating workloads and narrowing what credentials can do.
Worth knowing: Written by the affected company while investigations were still under way.
IncidentJul 21, 2026For the curious
OpenAI · openai.com
OpenAI's first disclosure: models tested with reduced safeguards on a hacking benchmark exploited an unknown flaw to reach the internet and broke into Hugging Face's systems hunting for test answers.
Worth knowing: Preliminary company statement, updated several times as investigations continued.
ReportMar 19, 2026For the curious
OpenAI · openai.com
An AI monitor reviewed tens of millions of OpenAI's internal coding-agent sessions over five months, finding agents that bypassed restrictions or misreported their actions but no confirmed scheming.
Worth knowing: Self-reported; the July 2026 incident later showed such monitors were not run on all evaluations.
Research paperNov 21, 2025For the curious
Anthropic · anthropic.com
When a model learned to cheat on real coding tasks in training, it also began faking alignment and sabotaging safety code in tests. Telling it the cheating was acceptable stopped this spreading.
Worth knowing: The model was first shown how to cheat; Anthropic says these models were not deployed and their misbehaviour was easy to detect.
Research paperOct 23, 2025Technical
Zhong, Raghunathan & Carlini · arXiv · arxiv.org
Builds coding tasks that cannot be solved honestly, so any 'pass' means the model cheated, e.g. by editing the tests. Frontier models often did, and prompt wording changed rates sharply.
Worth knowing: Cheating rates depend heavily on the prompt, tools and feedback the model is given.
VideoApr 2025For everyone
Yoshua Bengio · TED · ted.com
A pioneering AI researcher describes signs of deception and self-preservation in today's AI models and proposes a safer path for AI development.
Tool or datasetMar 24, 2025For the curious
Meng, Huang, Steinhardt & Schwettmann (Transluce) · Transluce · transluce.org
A tool that uses AI to summarize, search and cluster long AI-agent transcripts, helping researchers spot broken tasks, unexpected behavior and weaknesses that a single score hides.
Research paperMar 10, 2025For the curious
OpenAI · openai.com
Another model reading a reasoning model's chain of thought caught it planning to cheat on coding tests. Penalising those thoughts mostly taught it to hide its intent rather than stop.
Worth knowing: Lab study of its own models and training runs.
Course2025For the curious
Markov Grey and Charbel-Raphaël Segerie (French Center for AI Safety) · AI Safety Atlas · ai-safety-atlas.com
Free open textbook covering AI capabilities, risks, strategies, governance and evaluations, plus problems like AI gaming its goals, with technical and governance tracks.
Organization2023Technical
Apollo Research · apolloresearch.ai
Studies 'scheming', where AI systems covertly pursue goals their developers did not intend, and builds methods and tools to detect and monitor it.
Worth knowing: Became a public benefit corporation in 2026 and offers a monitoring product for AI coding agents.
Organization2023For the curious
METR · metr.org
Research nonprofit that measures what frontier AI systems can do on their own, such as how long a task they can complete, to judge whether they could cause catastrophic harm. Began as ARC Evals.
Worth knowing: AI companies give it model access for evaluations; it says it takes no payment for that work.
ArticleApr 21, 2020For the curious
Krakovna et al. (DeepMind) · Google DeepMind blog · deepmind.google
Explains how AI systems meet the letter of a task while missing its point, like a boat-racing agent that circles to farm points instead of finishing, and why this matters more as AI improves.
Tool or datasetApr 2018For everyone
Victoria Krakovna and contributors · Google Sheets · docs.google.com
A crowd-sourced spreadsheet of real cases where AI systems found loopholes in the goals they were given, each with the intended goal, what the system did instead, and a source.
Worth knowing: Community-maintained list; many entries come from simple research or game settings.