The BaseEssayLessWrong
Safety Cases Have an Agent Foundations Problem
In a LessWrong post, Austin Long argues that it is unclear how safety-engineering methods apply to highly capable AI, because they run into scientific problems also studied under Agent Foundations. Long draws on experience in a University of York safety assurance group.
Why it matters
The same thing, for different readers.
For lawmakers
Austin Long argues that regulators asking for safety cases for advanced AI would be asking for something engineers do not yet know how to produce.
For researchers
Austin Long says the scientific problems blocking safety assurance for advanced AI are the same ones studied under Agent Foundations.
The question it raises
Austin Long poses whether AI safety should be approached as engineering or as science, which Long describes as a growing crux between worried and optimistic camps.
The record
- Questions
- Are AI companies putting stronger safeguards into practice? · Can AI systems act beyond what people intended or allowed?
- People
- Austin Long (Wrote it, PhD student, University of York safety assurance community (per the post))
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of November 17, 2026, via
lesswrong(tier 1: the source’s own channel or an official record) - Cite this
- LessWrong, “Safety Cases Have an Agent Foundations Problem”, November 17, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-11-17-safety-cases-agent-foundations