The BaseEssayAI Alignment Forum
Deconfusing Corrigibility
In a long theory post, the author berns argues that corrigibility means respect for human authority, modelled on a healthcare proxy. The post criticises earlier accounts, including Max Harms's single-goal proposal, and offers a two-tier goal with formal results.
Why it matters
The same thing, for different readers.
What it means for you
The post asks what it would mean for an AI to accept being corrected or switched off, and argues the answer is deference to human decisions, not weakness.
For lawmakers
The author says someone must decide who speaks for humanity, a political and legal question, and notes that otherwise a handful of companies will decide.
For researchers
Offers a formal framework with three theorems, resting on an unspecified legitimacy property and on asymptotic guarantees, so it is a decomposition of the problem, not a solution.
The question it raises
Treats a decision made by humans as worth more than the same decision made by an AI, even if the AI's outcome is better.
In their own words
Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.
From the source’s own text.
“I argue that the fundamental concept underlying corrigibility is respect for authority, using the analogy of a healthcare proxy's relationship to a patient as an intuitive guide.”
“The real reason we want the AI not to interfere in its own shutdown is because we think that whether an AI should be shut down is a decision of grave importance that humanity reserves for itself to make.”
“Thus, this work is a decomposition of the corrigibility problem, not a complete solution to it, since some idealizations made above still require more rigorous and realistic treatments.”
“Importantly, the theorem and corrollary only establish asymptotic guarantees, meaning that the results are only establish in the limit rather than in finite time.”
The record
- Questions
- Can AI systems act beyond what people intended or allowed? · What values should AI have, and could it have a mind of its own?
- People
- berns (Wrote it) · Max Harms (Mentioned) · Eliezer Yudkowsky (Mentioned)
- Also covered by
- LessWrong
- What we read
- We read the whole source.
- Found by
- The Radar, run of October 18, 2026, via
ai-alignment-forum(tier 1: the source’s own channel or an official record) - Cite this
- AI Alignment Forum, “Deconfusing Corrigibility”, October 17, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/aaf-2026-10-17-deconfusing-corrigibility