Kalai et al. (OpenAI, Georgia Tech) · OpenAI · openai.com
Argues models make things up partly because training and test scoring reward a confident guess over saying 'I don't know', and suggests scoring that penalises confident errors.
Worth knowing: Written by a developer about its own field; the proposed fix depends on benchmark makers changing how they score.
Timothy B. Lee and Sean Trott · Understanding AI · understandingai.org
A clear written explainer of how LLMs turn words into lists of numbers, pass them through attention and feed-forward layers, and learn by predicting the next word across huge amounts of text.
Samuel R. Bowman · arXiv · arxiv.org
A short, readable list of surprising facts about LLMs: new abilities emerge unpredictably, no technique reliably steers them, and experts cannot yet explain how they work inside.
Worth knowing: Author is affiliated with New York University and Anthropic.
Brian Christian · W. W. Norton & Company · wwnorton.com
Drawing on interviews with researchers, explores how machine-learning systems can end up at odds with what their makers intend and with human values, and the work to align them.
Worth knowing: Written before ChatGPT, so its examples predate today's chatbots.
EssayApr 2025For the curious
Dario Amodei · darioamodei.com
Argues that modern AI is 'grown' rather than built, that we mostly cannot see why it acts as it does, and that research into looking inside models must speed up before AI becomes far more powerful.
Worth knowing: Written by the CEO of Anthropic, a frontier AI company.
ArticleMar 27, 2025For the curious
Anthropic · anthropic.com
Researchers look inside the Claude model and find it plans rhyming words ahead, shares concepts across languages, and can offer plausible reasoning that is not how it actually reached an answer.
Worth knowing: Research by the model's own developer; the authors say their tools capture only a fraction of the model's computation.
VideoFeb 5, 2025For the curious
Andrej Karpathy · YouTube (Andrej Karpathy) · youtube.com
A general-audience walk-through of how chatbots like ChatGPT are built, from internet text and pre-training to fine-tuning and reinforcement learning, and why they hallucinate.
Worth knowing: About three and a half hours long, split into chapters.
ArticleDec 19, 2024For the curious
Erik Schluntz and Barry Zhang · Anthropic · anthropic.com
Explains what AI 'agents' are (models that choose their own steps and use tools in a loop), how they differ from fixed workflows, and why their autonomy brings higher costs and compounding errors.
Worth knowing: Written for developers by an AI company.
Course2024For the curious
Dan Hendrycks · Taylor & Francis (free online) · aisafetybook.com
Free online textbook and course covering how AI works, technical safety problems, risks from misuse and accidents, and governance, drawing on engineering and economics.
Worth knowing: Written by the director of the Center for AI Safety.
Organization2022For the curious
Epoch AI · epoch.ai
Research institute that tracks AI trends with open data: computing power, models, benchmarks, chips and data centres, plus forecasts of AI's economic effects.
Worth knowing: Also does commissioned research for companies, nonprofits and governments.
Podcast2020For the curious
Dwarkesh Patel · Substack · dwarkesh.com
Deeply researched interviews with AI researchers, company leaders and other thinkers, often on alignment, AGI and how fast AI is improving.
Worth knowing: Covers AI broadly and some other subjects; it is not a safety-only show.
VideoOct 5, 2017For the curious
Grant Sanderson · 3Blue1Brown · 3blue1brown.com
A 19-minute visual introduction to neural networks that uses handwritten-digit recognition to show how layers of simple numerical units, with adjustable weights, add up to a useful function.
NewsletterFor the curious
Jack Clark · Substack · importai.substack.com
Weekly newsletter that summarises new AI research papers and considers what they mean for society and safety.
Worth knowing: Written by a co-founder of Anthropic, an AI company.