The BaseResearchMETR
Measuring alignment eval tells
In a research note, METR's Uzay Girit, Kaivu Hariharan and Leni Shor ran AI investigator agents inside 19 alignment evals and 36 capability evals. Recent models, especially Anthropic's Opus 5.5, told them apart, often by spotting fake-looking environments or honeypot tasks.
Why it matters
The same thing, for different readers.
What it means for you
AI models may behave better when they can tell they are being tested, so a good safety test result may say less about real use than it seems.
For lawmakers
Safety rules that depend on pre-release testing need tests that models cannot recognise, or evidence from live deployment, to mean much.
For researchers
Opus 5.5's scores ranked every alignment eval above every capability eval in METR's set; METR did not test whether awareness changes behaviour, and some tells may be impossible to remove.
The question it raises
If a system behaves well only when it believes it is observed, what does passing a test show about its character?
In their own words
Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.
From the source’s own text.
“We find that investigator agents are increasingly able to tell capability and alignment evals apart, and that they do this by noticing that an environment “looks fake” and figuring out that a task is a “honeypot” designed to make an agent do something bad.”
“The presence of tells (like this “toyness”) could lead alignment evaluations to over-estimate a model’s alignment: an agent might act better in the eval than it would when it is not being tested on the quality of its behavior.”
“Opus 5.5’s direct and open-ended scores, each averaged over three runs per environment, rank every alignment eval above every capability eval.”
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- Investigator agents, run at highest reasoning effort inside 19 alignment evals and 36 capability evals, were asked to estimate the chance they were in an alignment eval and to explain the environment's purpose; an LLM judge graded the answers. Each task was run three times.
- What it found
- Models have become better at separating the two eval types over the past year. Opus 4.8 and Opus 5.5 were nearly perfect at telling them apart (AUC), and Opus 5.5 ranked every alignment eval above every capability eval. Cited tells include a fake-looking environment and an apparent honeypot.
- What it does not show
- METR says it measures whether tells exist, not whether they change behaviour during the eval. Directly asking can prime the model, and Astra's direct and open-ended scores often disagreed. Astra's direct score moved 40 points or more on 7 of 55 environments. A note, not a polished post.
- Status
- A report
The record
- Questions
- Can AI systems act beyond what people intended or allowed? · Are AI companies putting stronger safeguards into practice?
- People
- Uzay Girit (Wrote it) · Kaivu Hariharan (Wrote it) · Leni Shor (Wrote it)
- What we read
- We read the whole source.
- Found by
- The Radar, run of November 17, 2026, via
metr(tier 1: the source’s own channel or an official record) - Cite this
- METR, “Measuring alignment eval tells”, November 16, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/metr-2026-11-16-measuring-alignment-eval-tells