The BaseResearcharXiv
Why Static Probes Fail: Context Determines Deception Feature Composition
Yao Huang and co-authors argue that detecting AI deception with a single linear probe in a model's activations does not generalize across settings. They propose a multi-direction method that adapts to each setting from about 10 examples, and report better detection than single-probe baselines on two open-weight models.
Why it matters
The same thing, for different readers.
What it means for you
Researchers report that one simple internal check for an AI lying works unevenly across situations, and propose a more flexible detector.
For lawmakers
Officials hoping for a standard internal check for AI deception should note the authors' finding that such detectors need retuning with labelled examples when the setting changes.
For researchers
The authors report that a single deception direction in a model's activations breaks across datasets, so interpretability teams relying on one probe may overstate how well deception detection carries over.
In their own words
Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.
From the source’s own text.
“such single-direction formulations generalize poorly across datasets and suffer substantial performance degradation when applied to heterogeneous deceptive scenarios, indicating that internal representations of deception are not universally aligned across domains.”
“with just 10 adaptation examples, it achieves an average AUROC of 0.97 and a recall of 95% at a 5% FPR across evaluated datasets, significantly outperforming existing static probing methods.”
“Although Qwen3-32B shows increased confusion on transaction-related insider scenarios due to partial representation overlap, our approach still surpasses all baselines by a substantial margin.”
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- Linear deception probes are compared with the authors' Dynamic Deception Subspace Navigation (several orthogonal directions combined by a gating network adapted from about 10 labelled examples per domain) on seven deception datasets, using Llama-3.3-70B-Instruct and Qwen3-32B, with black-box GPT-4o judges as extra baselines.
- What it found
- The authors report average AUROC of 0.97 on Llama-3.3-70B-Instruct and 0.93 on Qwen3-32B, with a mean recall of 94% at a 5% false-positive calibration, and say single-direction probes degrade on heterogeneous datasets such as MASK, PKU-DeceptionBench and THU-DeceptionBench.
- What it does not show
- Authors say Qwen3-32B is weaker on insider-trading scenarios. The method needs about 10 labelled examples per new domain, and few-shot adaptation was infeasible for some datasets with very long inputs. Only two open-weight models were tested.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Can AI systems act beyond what people intended or allowed? · Are AI companies putting stronger safeguards into practice?
- People
- Yao Huang (Wrote it) · Siyuan Chen (Wrote it) · Zhenghan Zhou (Wrote it) · Yinpeng Dong (Wrote it)
- What we read
- We read the whole source.
- Found by
- The Radar, run of October 21, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “Why Static Probes Fail: Context Determines Deception Feature Composition”, October 20, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-10-20-static-probes-fail-deception-subspace