The BaseResearcharXiv
Activation Probes Are Reliable With a Handful of Positive Examples
According to the abstract, Riya Tyagi and Stefan Heimersheim test activation probes for spotting rare misalignment when only a few positive examples exist. On spam and honesty tasks, training with many negatives beat balanced training for 1 to 10 positives, and larger models were more sample-efficient.
Why it matters
The same thing, for different readers.
For lawmakers
Riya Tyagi and Stefan Heimersheim suggest monitoring AI for rare misbehaviour can work with few known bad examples, which is relevant to what monitoring rules could realistically demand.
For researchers
The authors find that LLM upsampling is worth roughly doubling real positives, excessive upsampling hurts, and larger models are more positive-sample-efficient, on spam and honesty tasks only.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- Activation probes trained in a few-positives, thousands-of-negatives regime on spam and honesty detection tasks, varying class balance, LLM upsampling and model size.
- What it found
- For small numbers (1-10) of positives, training with many negatives was on average more positive-sample-efficient than balanced training; LLM upsampling gave a boost equivalent to roughly doubling real positives; larger models were more efficient.
- What it does not show
- Results are for the authors' spam and honesty tasks; they do not show the same holds for real misalignment.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Are AI companies putting stronger safeguards into practice? · Can AI systems act beyond what people intended or allowed?
- People
- Riya Tyagi (Wrote it) · Stefan Heimersheim (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of October 30, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “Activation Probes Are Reliable With a Handful of Positive Examples”, October 29, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-10-29-activation-probes-few-positive-examples