The BaseResearcharXiv
Beyond Causal Masks: Auditing and Localizing Hidden Future-Token Leakage in Adaptive Sequence Models
According to the abstract, a preprint shows how an adaptive routing decoder looked far better in validation, with loss of 1.4259 against 3.5958, because of hidden future-token leakage despite a correct causal mask. Two repaired models removed the apparent advantage.
Why it matters
The same thing, for different readers.
For researchers
A reminder that large apparent architecture gains can come from evaluation leaks, and that a prefix-invariance audit can detect them.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A 24-layer anomalous checkpoint was audited with prefix-invariance tests, LeakScore and layer-wise diagnostics; two independently trained 12-layer causal repairs were compared, with six synthetic implementation pairs as support.
- What it found
- Validation loss fell from 3.5958 to 1.4259 in the anomalous checkpoint, which failed the audit; the repairs removed detectable leakage and the advantage.
- What it does not show
- Abstract only was read; a case study of one architecture family.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- How fast is AI advancing, and where is it heading?
- People
- Huiting Liu (Wrote it) · Jinsong Wu (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of October 23, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “Beyond Causal Masks: Auditing and Localizing Hidden Future-Token Leakage in Adaptive Sequence Models”, October 22, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-10-22-hidden-future-token-leakage
Our notesLight lane: written from the triage excerpt only. Marginal fit for the Base.