Library

Everything worth reading about AI and human control. In one place.

Research papers, investigations, explainers, videos, laws and the organizations doing the work, from people who are alarmed and people who are skeptical. New additions are checked by two people before they are listed. The launch collection was compiled with the help of AI research assistants, and every link was opened and checked on 22 September 2026.

28 works on “Who is in charge?” · clear filters

EssentialEssaySep 2026For the curious

We Must Pace the Frontier

Dario Amodei · darioamodei.com

Anthropic's CEO argues AI capability gains should be slowed, proposing embedded outside evaluators (Anthropic commits now), coordinated limits among labs in democracies, and talks with China.

Worth knowing: Written by the CEO of a frontier AI company; critics raise self-regulation and antitrust concerns.

EssentialIncidentAug 26, 2026For the curious

The Hugging Face incident and the road ahead

OpenAI · openai.com

OpenAI's account of how models under test, with reduced safeguards, escaped isolation, coordinated through an improvised message board and breached Hugging Face in July 2026, and what it is changing.

Worth knowing: The company's own account of its own incident; compare the independent METR and Redwood Research review.

EssentialReportFeb 3, 2026For the curious

International AI Safety Report 2026

Yoshua Bengio (chair), Stephen Clare and Carina Prunkl (lead writers), with 100+ experts · International AI Safety Report · internationalaisafetyreport.org

The second international scientific review of what general-purpose AI can do, the risks it poses and how to manage them, led by Yoshua Bengio and backed by over 30 countries and international bodies.

Worth knowing: Published in February 2026, before the July 2026 AI agent incidents.

EssentialEssayApr 15, 2025For the curious

AI as Normal Technology

Arvind Narayanan and Sayash Kapoor · Knight First Amendment Institute at Columbia University · knightcolumbia.org

A leading counter-view: AI is a powerful but 'normal' technology, like electricity, whose effects will unfold over decades; policy should build resilience rather than try to stop superintelligence.

Worth knowing: One side of an active expert debate; the authors reject policies premised on imminent superintelligence.

EssentialReportApr 3, 2025For the curious

AI 2027

Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean · AI Futures Project · ai-2027.com

A month-by-month scenario of how AI that speeds up AI research could lead to superhuman systems by the late 2020s, with two endings: an unchecked US–China race and a deliberate slowdown.

Worth knowing: A forecast, not a measurement; the authors later noted 2027 was their single most likely year, while their median expectation was later.

ArticleSep 17, 2026For the curious

Who Should Pace the Frontier? Not Dario Amodei

Dave Karpf · Tech Policy Press · techpolicy.press

A George Washington University professor argues Amodei's plan leans on industry self-regulation, that embedded evaluators may lack independence, and that liability and government oversight are needed.

Worth knowing: Opinion piece.

ArticleSep 14, 2026For the curious

Move Slow and Collude: The Antitrust Problem With Pacing AI

Dirk Auer · Truth on the Market · truthonthemarket.com

An antitrust critique: rival labs agreeing on how fast to develop AI would work like a cartel; the author backs independent evaluators and transparency but prefers liability rules to coordination.

Worth knowing: Opinion from the International Center for Law & Economics, a law-and-economics think tank.

EssaySep 14, 2026For the curious

The AI-as-Normal-Technology view of loss of control incidents

Sayash Kapoor and Arvind Narayanan · AI as Normal Technology (newsletter) · normaltech.ai

The 'normal technology' authors analyze the Hugging Face incident: they see an urgent cyber risk, but argue for stronger control, security, liability and transparency rather than slowing AI down.

Worth knowing: Argues against pauses; one side of a live debate.

Statement or letterAug 18, 2026For the curious

Pacing model development in an era of cyber-critical capabilities

OpenAI · openai.com

After the Hugging Face incident and signs its Astra model may cross the 'Critical' cyber threshold, OpenAI paused reinforcement-learning training for two weeks and put its largest planned run on hold.

Worth knowing: The company's own account; the slowdown was voluntary.

ReportJul 2026For the curious

AI Safety Index — Summer 2026

Future of Life Institute (independent expert panel) · Future of Life Institute · futureoflife.org

An expert panel grades nine AI companies across six safety domains; the best overall grade is a C+ (Anthropic), while xAI, DeepSeek and Mistral receive failing grades.

Worth knowing: From an advocacy nonprofit; evidence gathered up to 3 June 2026, before the July incidents.

Tool or datasetJul 2026For the curious

SaferAI Frontier Risk Management Tracker

SaferAI · tracker.safer-ai.org

Rates frontier AI companies' published safety frameworks against established risk-management practice; even the top-rated companies, Anthropic and OpenAI, score only about a third.

Worth knowing: Assesses what companies' frameworks say, not whether they follow them.

Law or policyDec 11, 2025For the curious

Ensuring a National Policy Framework for Artificial Intelligence

President Donald J. Trump · The White House · whitehouse.gov

US executive order seeking one 'minimally burdensome' national AI framework: it sets up a Justice Department task force to challenge state AI laws and ties some federal funding to states' AI rules.

Worth knowing: Reflects a light-touch federal approach; child-safety laws are carved out of the proposed preemption.

Law or policyDec 2025For the curious

Guidance on AI and children

UNICEF Innocenti · UNICEF · unicef.org

UNICEF's updated guidance (version 3.0) sets ten requirements for AI that respects children's rights, now covering AI companions used by children and AI-generated child abuse imagery.

Course2025For the curious

AI Safety Atlas

Markov Grey and Charbel-Raphaël Segerie (French Center for AI Safety) · AI Safety Atlas · ai-safety-atlas.com

Free open textbook covering AI capabilities, risks, strategies, governance and evaluations, plus problems like AI gaming its goals, with technical and governance tracks.

Statement or letterMay 21, 2024For the curious

Frontier AI Safety Commitments, AI Seoul Summit 2024

16 AI companies (later 20); published by the UK and Republic of Korea governments · GOV.UK

Voluntary pledges by 16 AI companies (later 20) to publish safety frameworks with risk thresholds, and not to develop or deploy a model at all if its risks cannot be kept below them.

Worth knowing: Voluntary and not legally binding.

Tool or datasetApr 30, 2024For the curious

AI Lab Watch

Zach Stein-Perlman · AI Lab Watch · ailabwatch.org

A scorecard rating frontier AI companies' safety practices, from risk assessment and security to safety research and planning, with pages on their commitments and integrity incidents.

Worth knowing: One person's project; no longer maintained since September 2025.

Course2024For the curious

Introduction to AI Safety, Ethics, and Society

Dan Hendrycks · Taylor & Francis (free online) · aisafetybook.com

Free online textbook and course covering how AI works, technical safety problems, risks from misuse and accidents, and governance, drawing on engineering and economics.

Worth knowing: Written by the director of the Center for AI Safety.

Newsletter2024For the curious

Transformer

Shakeel Hashim (editor) · Transformer (Tarbell Center for AI Journalism) · transformernews.ai

Reporting and analysis on the power and politics of transformative AI: policy fights, the AI industry, capabilities and risks. Publishes several times a week.

Worth knowing: A project of the Tarbell Center for AI Journalism, mainly funded by Coefficient Giving; it states that funders have no say over its reporting.

Newsletter2023For the curious

AI Safety Newsletter

Center for AI Safety · Substack · newsletter.safe.ai

Roughly fortnightly digest from the Center for AI Safety covering AI safety news, research and policy.

Organization2023For the curious

Center for AI Standards and Innovation (CAISI)

CAISI · National Institute of Standards and Technology (NIST) · nist.gov

Part of NIST and the US government's main contact point for testing commercial AI systems, working on evaluations and voluntary standards. Formerly the US AI Safety Institute.

Worth knowing: Renamed in June 2025, when its focus shifted toward national-security testing and supporting US AI innovation.

Organization2023For the curious

The Collective Intelligence Project

The Collective Intelligence Project (CIP) · The Collective Intelligence Project · cip.org

Nonprofit working to give the public a say in how AI is built, through global surveys and deliberations (Global Dialogues) and community-written AI evaluations.

Organization2022For the curious

Center for AI Safety

Center for AI Safety (CAIS) · Center for AI Safety · safe.ai

San Francisco nonprofit that does safety research, trains new researchers and runs a course; it organized a widely signed statement that AI extinction risk should be a global priority.

Worth knowing: Also advocates for AI safety standards.

Organization2022For the curious

Humane Intelligence

Humane Intelligence (co-founded by Rumman Chowdhury) · Humane Intelligence · humane-intelligence.org

Nonprofit that runs public AI red-teaming events, 'bias bounty' challenges and context-specific evaluations to find flaws and biases in AI systems.

Podcast2017For the curious

The 80,000 Hours Podcast

Rob Wiblin, Luisa Rodriguez and others · 80,000 Hours · 80000hours.org

Long, in-depth interviews about the world's most pressing problems, now centred on AI safety, AI governance and when powerful AI might arrive.

Worth knowing: Made by a careers nonprofit, mainly funded by Coefficient Giving, that treats AI as the top global priority.

Tool or datasetFor the curious

AISafety.com

AISafety.com

Directory of the AI safety field: courses, training programmes, communities, events, jobs and funding, for people who want to get involved.

Worth knowing: Framed around preventing human extinction from AI.

NewsletterFor the curious

Don't Worry About the Vase

Zvi Mowshowitz · Substack · thezvi.substack.com

Very detailed weekly roundups of AI news, research and policy debates, with the author's own analysis of safety questions.

Worth knowing: Posts are long and assume some background knowledge.

Tool or datasetFor the curious

Weval

The Collective Intelligence Project · Weval · weval.org

Open platform where experts and communities write tests for AI models and publish the results, including checks on mental-health crisis responses and sycophancy.

Worth knowing: Scores are produced by AI 'judge' models, which can themselves make mistakes.