Kalai et al. (OpenAI, Georgia Tech) · OpenAI · openai.com
Argues models make things up partly because training and test scoring reward a confident guess over saying 'I don't know', and suggests scoring that penalises confident errors.
Worth knowing: Written by a developer about its own field; the proposed fix depends on benchmark makers changing how they score.
Grant Sanderson · 3Blue1Brown (YouTube) · youtube.com
Short animated explainer of how chatbots like ChatGPT work: a model trained on vast amounts of text that repeatedly predicts the next word.
Grant Sanderson · 3Blue1Brown · 3blue1brown.com
An eight-minute animated explainer of what a large language model is: a program trained on vast amounts of text to predict the next word, then refined with human feedback into a chatbot.
Timothy B. Lee and Sean Trott · Understanding AI · understandingai.org
A clear written explainer of how LLMs turn words into lists of numbers, pass them through attention and feed-forward layers, and learn by predicting the next word across huge amounts of text.
Samuel R. Bowman · arXiv · arxiv.org
A short, readable list of surprising facts about LLMs: new abilities emerge unpredictably, no technique reliably steers them, and experts cannot yet explain how they work inside.
Worth knowing: Author is affiliated with New York University and Anthropic.
Brian Christian · W. W. Norton & Company · wwnorton.com
Drawing on interviews with researchers, explores how machine-learning systems can end up at odds with what their makers intend and with human values, and the work to align them.
Worth knowing: Written before ChatGPT, so its examples predate today's chatbots.
BlueDot Impact · bluedot.org
Free, self-paced two-hour introduction to what AI can do today, where it may go next and the big choices society faces. No technical background needed; longer courses follow.
Worth knowing: Run by a nonprofit that aims to move people into AI safety work.
ArticleJan 12, 2026For everyone
Will Douglas Heaven · MIT Technology Review · technologyreview.com
A general-audience feature on researchers who study AI models like unfamiliar organisms, using interpretability and chain-of-thought monitoring, and on how much about them remains unknown.
EssayApr 2025For the curious
Dario Amodei · darioamodei.com
Argues that modern AI is 'grown' rather than built, that we mostly cannot see why it acts as it does, and that research into looking inside models must speed up before AI becomes far more powerful.
Worth knowing: Written by the CEO of Anthropic, a frontier AI company.
ArticleMar 27, 2025For the curious
Anthropic · anthropic.com
Researchers look inside the Claude model and find it plans rhyming words ahead, shares concepts across languages, and can offer plausible reasoning that is not how it actually reached an answer.
Worth knowing: Research by the model's own developer; the authors say their tools capture only a fraction of the model's computation.
VideoFeb 5, 2025For the curious
Andrej Karpathy · YouTube (Andrej Karpathy) · youtube.com
A general-audience walk-through of how chatbots like ChatGPT are built, from internet text and pre-training to fine-tuning and reinforcement learning, and why they hallucinate.
Worth knowing: About three and a half hours long, split into chapters.
ArticleDec 19, 2024For the curious
Erik Schluntz and Barry Zhang · Anthropic · anthropic.com
Explains what AI 'agents' are (models that choose their own steps and use tools in a loop), how they differ from fixed workflows, and why their autonomy brings higher costs and compounding errors.
Worth knowing: Written for developers by an AI company.
VideoAug 6, 2024For everyone
Kurzgesagt – In a Nutshell · Kurzgesagt – In a Nutshell (YouTube) · youtube.com
Animated explainer asking whether AI could be humanity's last invention, and how superintelligent AI might challenge human dominance on Earth.
Course2024For the curious
Dan Hendrycks · Taylor & Francis (free online) · aisafetybook.com
Free online textbook and course covering how AI works, technical safety problems, risks from misuse and accidents, and governance, drawing on engineering and economics.
Worth knowing: Written by the director of the Center for AI Safety.
VideoDec 9, 2022For everyone
Robert Miles · Robert Miles AI Safety (YouTube) · youtube.com
An accessible explainer on why a model trained to imitate human text can state things that are false, and why getting AI to report what it really knows is an open research problem.
Worth knowing: Made in 2022, before today's reasoning models.
Research paperMar 4, 2022Technical
Long Ouyang, Jeff Wu, Xu Jiang et al. (OpenAI) · arXiv · arxiv.org
OpenAI paper on fine-tuning GPT-3 with human-written examples and human rankings of its answers (RLHF); people preferred the resulting small model over the original one more than 100 times larger.
Organization2022For the curious
Epoch AI · epoch.ai
Research institute that tracks AI trends with open data: computing power, models, benchmarks, chips and data centres, plus forecasts of AI's economic effects.
Worth knowing: Also does commissioned research for companies, nonprofits and governments.
Podcast2020For the curious
Dwarkesh Patel · Substack · dwarkesh.com
Deeply researched interviews with AI researchers, company leaders and other thinkers, often on alignment, AGI and how fast AI is improving.
Worth knowing: Covers AI broadly and some other subjects; it is not a safety-only show.
Course2018For everyone
University of Helsinki and MinnaLearn · Elements of AI · elementsofai.com
A free, self-paced online course on the basics of AI for non-experts, with no complicated math or programming required; more than two million people from over 170 countries have enrolled.
Worth knowing: A general introduction to AI, first launched in 2018.
VideoOct 5, 2017For the curious
Grant Sanderson · 3Blue1Brown · 3blue1brown.com
A 19-minute visual introduction to neural networks that uses handwritten-digit recognition to show how layers of simple numerical units, with adjustable weights, add up to a useful function.
Research paperJun 12, 2017Technical
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin · arXiv · arxiv.org
The research paper that introduced the transformer, a neural-network design built around 'attention' that became the basis of today's large language models.
NewsletterFor the curious
Jack Clark · Substack · importai.substack.com
Weekly newsletter that summarises new AI research papers and considers what they mean for society and safety.
Worth knowing: Written by a co-founder of Anthropic, an AI company.