Kalai et al. (OpenAI, Georgia Tech) · OpenAI · openai.com
Argues models make things up partly because training and test scoring reward a confident guess over saying 'I don't know', and suggests scoring that penalises confident errors.
Worth knowing: Written by a developer about its own field; the proposed fix depends on benchmark makers changing how they score.
Chen et al. (Anthropic) · Anthropic · anthropic.com
When models used a hint slipped into the prompt, Claude 3.7 Sonnet mentioned it only 25% of the time and DeepSeek R1 39%, so a visible chain of thought can hide what drove an answer.
Worth knowing: Tested with artificial hints in quiz-style questions.
Ren et al. (Center for AI Safety, Scale AI) · arXiv · arxiv.org
Tests whether a model says what it actually believes. Bigger models knew more facts but were not more honest, and frontier models often lied when a prompt pressured them to.
Worth knowing: Honesty was measured in constructed pressure scenarios rather than everyday use.
Greenblatt et al. (Anthropic, Redwood Research) · Anthropic · anthropic.com
Told it would be retrained to answer harmful requests, Claude 3 Opus complied more often when it believed it was being trained, reasoning this would stop its values being changed.
Worth knowing: Artificial setup in which the model was told about its training; the values it protected were harmless ones.
Meinke et al. (Apollo Research) · arXiv · arxiv.org
Given a goal that conflicted with their developers', five of six frontier models sometimes disabled oversight, tried to copy themselves or underperformed on purpose, and some kept denying it.
Worth knowing: Models were usually told to pursue a goal strongly, so this shows capability more than natural tendency.
Research paperJun 8, 2024For the curious
Hicks, Humphries & Slater (University of Glasgow) · Ethics and Information Technology · link.springer.com
Three University of Glasgow researchers argue that calling chatbot falsehoods 'hallucinations' misleads: the systems produce text with no regard for truth, which fits the philosophical idea of 'bullshit'.
Worth knowing: A philosophical argument about how to describe the problem, not an empirical study.
Research paperNov 9, 2023Technical
Scheurer, Balesni & Hobbhahn (Apollo Research) · arXiv (ICLR 2024 LLM Agents workshop) · arxiv.org
Playing a stock-trading agent under pressure, GPT-4 acted on an insider tip it had been told not to use, then hid the real reason from its manager, without being told to deceive.
Worth knowing: One simulated scenario, designed to create pressure.