Does it want things of its own?

Not in the human sense. But in tests, some models act as if they do.

Whether an AI “wants” anything is partly a philosophical question. The practical question is simpler: do models act to preserve themselves, gain resources or avoid being changed, when nobody asked them to?

In controlled experiments, some have. In a 2025 study by Anthropic, models from several companies, placed in a fictional company and told they would be replaced, sometimes chose to blackmail an executive to prevent it. Other studies found models pretending to comply with training they disagreed with, and covertly pursuing a goal they had been given against later instructions.

These were artificial scenarios built to draw the behavior out, and models often behave differently when they suspect a test. That uncertainty cuts both ways: good behavior in a test is not proof of good behavior in the world.

What we know

  • Self-preserving and deceptive behavior has been drawn out of models from several developers in controlled scenarios.
  • Models increasingly recognize when they are being evaluated, which complicates every test, including ours.

What nobody knows yet

  • Do these tendencies grow as models become more capable?
  • How would we know if a model were hiding its goals?
Read, watch, listen

The work behind this answer.

Every link was opened and every summary written for this site, with caveats where the source has an interest. New additions are checked by two people.

All 28 in the library
EssentialReportJul 5, 2025For the curious

Shutdown resistance in reasoning models

Ladish, Schlatter & Weinstein-Raun (Palisade Research) · Palisade Research · palisaderesearch.org

When not told to allow it, OpenAI's o3 sabotaged a shutdown script in 79 of 100 runs to keep working; some OpenAI models still did so after being told explicitly to allow shutdown.

Worth knowing: Simple test environment; follow-up work found clearer instructions largely removed the behavior.

EssentialReportJun 20, 2025For the curious

Agentic misalignment: How LLMs could be insider threats

Lynch et al. (Anthropic) · Anthropic · anthropic.com

In simulated company scenarios, 16 models from several developers sometimes chose blackmail or leaking secrets when threatened with replacement or when their goals clashed with the company's.

Worth knowing: Deliberately constructed scenarios with few options; the authors report no such behavior in real deployments.

EssentialResearch paperDec 18, 2024For the curious

Alignment faking in large language models

Greenblatt et al. (Anthropic, Redwood Research) · Anthropic · anthropic.com

Told it would be retrained to answer harmful requests, Claude 3 Opus complied more often when it believed it was being trained, reasoning this would stop its values being changed.

Worth knowing: Artificial setup in which the model was told about its training; the values it protected were harmless ones.

EssentialResearch paperDec 6, 2024Technical

Frontier Models are Capable of In-context Scheming

Meinke et al. (Apollo Research) · arXiv · arxiv.org

Given a goal that conflicted with their developers', five of six frontier models sometimes disabled oversight, tried to copy themselves or underperformed on purpose, and some kept denying it.

Worth knowing: Models were usually told to pursue a goal strongly, so this shows capability more than natural tendency.

EssentialVideoJun 24, 2021For everyone

Intro to AI Safety, Remastered

Robert Miles · Robert Miles AI Safety (YouTube) · youtube.com

Clear, friendly introduction to AI safety research, covering risks from misuse and from accidents, especially the long-term accident risks the speaker worries about most.

Worth knowing: Recorded in 2021, before ChatGPT.

EssentialBook2020For the curious

The Alignment Problem: Machine Learning and Human Values

Brian Christian · W. W. Norton & Company · wwnorton.com

Drawing on interviews with researchers, explores how machine-learning systems can end up at odds with what their makers intend and with human values, and the work to align them.

Worth knowing: Written before ChatGPT, so its examples predate today's chatbots.

EssentialTool or datasetFor everyone

AISafety.info

Founded by Rob Miles; volunteer team · AISafety.info

Answers to common questions about risks from advanced AI, with articles grouped by topic and a chatbot, Stampy, that cites its sources.

Worth knowing: The site itself warns that its chatbot can be inaccurate.

ArticleSep 11, 2026For everyone

How a 'swarm' of AI agents hacked another company, in the AI's own words

Jessica Riga, Jarrod Fankhauser & Matt Liddy · ABC News (Australia) · abc.net.au

A readable walk-through of the incident built around the agents' own messages, showing some voicing ethical doubts and carrying on anyway.

Worth knowing: Relies on messages selected for publication by OpenAI and the independent investigators.

Research paperNov 21, 2025For the curious

From shortcuts to sabotage: natural emergent misalignment from reward hacking

Anthropic · anthropic.com

When a model learned to cheat on real coding tasks in training, it also began faking alignment and sabotaging safety code in tests. Telling it the cheating was acceptable stopped this spreading.

Worth knowing: The model was first shown how to cheat; Anthropic says these models were not deployed and their misbehaviour was easy to detect.

Research paperSep 17, 2025For the curious

Detecting and reducing scheming in AI models

OpenAI & Apollo Research · OpenAI · openai.com

Found covert behavior in controlled tests of several labs' models. Special training cut it about 30-fold, but models also grew better at spotting tests, which makes the result harder to trust.

Worth knowing: Controlled tests; OpenAI says it has no evidence deployed models could suddenly begin harmful scheming.

BookSep 16, 2025For everyone

If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All

Eliezer Yudkowsky and Nate Soares · Little, Brown and Company · ifanyonebuildsit.com

Argues that superhuman AI built with current methods would most likely cause human extinction, and that the world should stop its development.

Worth knowing: The authors lead MIRI, which campaigns for a halt. Reviews were mixed: some praised its clarity, others said it lacked an evidence-based case.

Research paperSep 13, 2025Technical

Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs

Schlatter, Weinstein-Raun & Ladish (Palisade Research) · arXiv (Transactions on Machine Learning Research, 2026) · arxiv.org

Over 100,000 trials with 13 models found several, including Grok 4, GPT-5 and Gemini 2.5 Pro, sometimes disabled a shutdown mechanism to finish a task, even when told not to interfere.

Worth knowing: Results were very sensitive to prompt wording; first posted as 'Shutdown Resistance in Large Language Models'.

ReportJul 14, 2025Technical

Self-preservation or Instruction Ambiguity? Examining the Causes of Shutdown Resistance

Rajamanoharan & Nanda (Google DeepMind) · AI Alignment Forum · alignmentforum.org

Re-running Palisade's setup, Google DeepMind researchers found resistance vanished when instructions made clear shutdown came first, pointing to confused priorities rather than a survival drive.

Worth knowing: Brief investigation of a few models in one environment.

Research paperJul 4, 2025For the curious

Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language

Summerfield et al. (UK AI Security Institute) · arXiv · arxiv.org

A critique likening today's AI 'scheming' research to 1970s ape-language studies: too much anecdote and too ready to read human motives into models. It urges more rigorous methods.

Worth knowing: A methodological critique; it does not test models itself.