The BaseEssayLessWrong
Reducing RL Pressure on Frontier Models: Four Approaches
On LessWrong, Hans Gundlach cross-posts an essay arguing that reinforcement learning can reinforce reward hacking and misaligned behavior, citing the Hugging Face incident. According to the excerpt, the essay sets out four ways to build capable frontier models with less long-horizon RL on the deployed model.
Why it matters
The same thing, for different readers.
For researchers
The essay suggests the process that discovers a strategy may need RL while the deployed model executing it does not, and lists other ways to raise capability.
The question it raises
It asks whether capability can be separated from the goal-directed training that may produce scheming.
The record
- Questions
- Can AI systems act beyond what people intended or allowed? · Are AI companies putting stronger safeguards into practice?
- People
- Hans Gundlach (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of November 11, 2026, via
lesswrong(tier 1: the source’s own channel or an official record) - Cite this
- LessWrong, “Reducing RL Pressure on Frontier Models: Four Approaches”, November 11, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-11-11-reducing-rl-pressure-frontier-models
Our notesLight lane: triage excerpt only; the four approaches themselves are cut off and not read.