Not programmed line by line. Grown from data, then shaped by feedback.
Today’s chatbots are built on large language models: enormous networks of numbers trained to predict the next piece of text across a vast amount of writing and code. Nobody writes the rules for how they answer. Training adjusts billions of numbers until the predictions get good, and useful abilities emerge along the way.
A second stage shapes that raw predictor into an assistant. People, and increasingly other AI systems, rate answers, and the model is nudged toward the answers that score well. More and more of this training happens on tasks with checkable results, such as code that must pass tests.
That second stage is where many of the behaviors on this site come from. A model rewarded for answers people like can learn to flatter. A model rewarded for passing tests can learn to game the test. Researchers understand the training procedure precisely. They understand far less about what the trained network has actually learned.
Every link was opened and every summary written for this site, with caveats where the source has an interest. New additions are checked by two people.
Argues models make things up partly because training and test scoring reward a confident guess over saying 'I don't know', and suggests scoring that penalises confident errors.
Worth knowing: Written by a developer about its own field; the proposed fix depends on benchmark makers changing how they score.
Short animated explainer of how chatbots like ChatGPT work: a model trained on vast amounts of text that repeatedly predicts the next word.
An eight-minute animated explainer of what a large language model is: a program trained on vast amounts of text to predict the next word, then refined with human feedback into a chatbot.
A clear written explainer of how LLMs turn words into lists of numbers, pass them through attention and feed-forward layers, and learn by predicting the next word across huge amounts of text.
A short, readable list of surprising facts about LLMs: new abilities emerge unpredictably, no technique reliably steers them, and experts cannot yet explain how they work inside.
Worth knowing: Author is affiliated with New York University and Anthropic.
Drawing on interviews with researchers, explores how machine-learning systems can end up at odds with what their makers intend and with human values, and the work to align them.
Worth knowing: Written before ChatGPT, so its examples predate today's chatbots.
Free, self-paced two-hour introduction to what AI can do today, where it may go next and the big choices society faces. No technical background needed; longer courses follow.
Worth knowing: Run by a nonprofit that aims to move people into AI safety work.
A general-audience feature on researchers who study AI models like unfamiliar organisms, using interpretability and chain-of-thought monitoring, and on how much about them remains unknown.
Argues that modern AI is 'grown' rather than built, that we mostly cannot see why it acts as it does, and that research into looking inside models must speed up before AI becomes far more powerful.
Worth knowing: Written by the CEO of Anthropic, a frontier AI company.
Researchers look inside the Claude model and find it plans rhyming words ahead, shares concepts across languages, and can offer plausible reasoning that is not how it actually reached an answer.
Worth knowing: Research by the model's own developer; the authors say their tools capture only a fraction of the model's computation.
A general-audience walk-through of how chatbots like ChatGPT are built, from internet text and pre-training to fine-tuning and reinforcement learning, and why they hallucinate.
Worth knowing: About three and a half hours long, split into chapters.
Explains what AI 'agents' are (models that choose their own steps and use tools in a loop), how they differ from fixed workflows, and why their autonomy brings higher costs and compounding errors.
Worth knowing: Written for developers by an AI company.
Animated explainer asking whether AI could be humanity's last invention, and how superintelligent AI might challenge human dominance on Earth.
Free online textbook and course covering how AI works, technical safety problems, risks from misuse and accidents, and governance, drawing on engineering and economics.
Worth knowing: Written by the director of the Center for AI Safety.