The BaseResearchLessWrongOctober 21, 2026

Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs

According to the excerpt, Davide Baldelli reports a MATS 10.0 study asking whether models can learn in context to control a score derived from their own activations, a test of whether probes that detect misbehaviour can be evaded. Across ten open-weight models, control is small and limited.

Open the source ↗

Why it matters

The same thing, for different readers.

For lawmakers

Monitoring a model through its internals looks somewhat sturdier here, though the post reports one model that could raise the score.

For researchers

Activation probes assume models cannot control what their activations encode; this test across ten models finds such control small, with one model able to raise a monitored score.

How the finding was reached

In our words, from the source. Weigh the finding by how it was made.

What was done: Ten open-weight models from five families (9B to 32B parameters) were shown a score derived from their own activations each turn and asked to raise or lower it, with a separate writer model as a control for task competence.
What it found: Some models move their scores considerably but no more than a writer of comparable competence; with the sentence fixed, three of ten show small consistent control; no model learns to lower a monitor's score and one can raise it.
What it does not show: Only the opening of the post was read; the authors' stated limits were not seen.
Status: A preprint: not yet peer reviewed

The record

Questions: Can AI systems act beyond what people intended or allowed? · Are AI companies putting stronger safeguards into practice?
People: Davide Baldelli (Wrote it) · Mirko Bronzi (Mentioned, Supervisor) · Damiano Fornasiere (Mentioned, Supervisor)
What we read: We have read only the description the publisher gives. No quotations until someone reads the source itself.
Found by: The Radar, run of October 22, 2026, via lesswrong (tier 1: the source’s own channel or an official record)
Cite this: LessWrong, “Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs”, October 21, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-10-21-cross-model-neurofeedback-activation-control

Our notesLight lane: written from the truncated excerpt of a post summarising a paper. No source text saved.