The BaseEssayRedwood Research

We don’t understand whether GDM intentionally trains on CoT

Redwood Research authors Alek Westover and Buck Shlegeris say Google DeepMind's Gemini 4 safety report appears to mention training that observes chains of thought, while a DeepMind blog says CoT is not penalised. They ask DeepMind to clarify and argue such training could suppress honest reasoning.

Open the source ↗3 quotations checked, word for word ↓

Why it matters

The same thing, for different readers.

What it means for you

Researchers outside Google DeepMind say they cannot tell from its safety report whether the company trains Gemini on the model's visible reasoning, a practice that could hide warning signs.

For lawmakers

Safety reports that leave a key training practice ambiguous limit what outside researchers and regulators can check; Redwood's authors call for AI companies to be more transparent.

For researchers

Redwood argues that reward signals which merely look 'orthogonal' to the output reward can still teach models to avoid verbalising misaligned thoughts, and shows judges rating edited transcripts higher.

The question it raises

If models learn to avoid saying what they intend, the honesty of their stated reasoning becomes a question about training pressure, not character.

In their own words

Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.

From the source’s own text.

“We’re pretty worried that these judges would be biased against models expressing misaligned thoughts, and otherwise disincentivize models reporting their thoughts honestly.”
Redwood ResearchExplaining why training on chains of thought with auxiliary rewards would concern the authors.
Found word for word in the sourceAs quoted in the text we read, not the speaker’s own text or a transcript of their words
“Their notion of orthogonality is extremely weak, so we are very skeptical that the claim is true in general.”
Redwood ResearchOn the definition of orthogonal CoT rewards in Kaufmann et al., the paper cited by the Gemini 4 safety report.
Found word for word in the sourceAs quoted in the text we read, not the speaker’s own text or a transcript of their words
“Our best guess is that somehow this is a misunderstanding and GDM isn’t actually intentionally training on the CoT in a particularly concerning way, and that even if they are it isn’t directly that bad for CoT faithfulness.”
Redwood ResearchThe authors' tentative conclusion, before asking for clarification from GDM.
Found word for word in the sourceAs quoted in the text we read, not the speaker’s own text or a transcript of their words

The record

Questions
Are AI companies putting stronger safeguards into practice? · Can AI systems act beyond what people intended or allowed?
People
Alek Westover (Wrote it) · Buck (Wrote it)
What we read
We read the whole source.
Found by
The Radar, run of November 11, 2026, via redwood-research-blog (tier 1: the source’s own channel or an official record)
Cite this
Redwood Research, “We don’t understand whether GDM intentionally trains on CoT”, November 10, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/redwood-2026-11-10-gdm-train-on-cot-unclear
Our notesJoint post by two authors, so quotations are attributed to the blog rather than one person. Their reading of the Gemini 4 safety report is their interpretation; they say GDM staff told them CoT is not trained on. Equations and figures not in the saved text. The story id is chosen by topic and should be confirmed.