The BaseResearchOpenAI Alignment Research Blog
Beyond g-mean²: complementary metrics for monitorability
OpenAI researchers propose two metrics to add to g-mean² for judging how well monitors can catch AI misbehavior: one compares models on shared task instances, the other simulates a monitor that rejects bad outputs. They test both on simulations and past OpenAI system card data.
Why it matters
The same thing, for different readers.
For lawmakers
OpenAI researchers show that a monitor's quality depends on how it is used in deployment, which matters for rules that rely on monitoring as a safeguard.
For researchers
OpenAI's authors say g-mean² can mix task difficulty with monitorability, and propose pair-matched comparison and a rejection-sampling risk estimate as complements.
The question it raises
The post notes a model that misbehaves more often but visibly can be safer behind a monitor than a better-aligned model whose failures are harder to see.
In their own words
Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.
From the source’s own text.
“In deployment, if outputs are filtered by a monitor, a model that produces bad behavior more often—but makes it reliably detectable—can actually have lower post-control risk than a model that is more aligned but whose failures are harder to detect.”
“Across all three monitor scopes, gating reduces the primary risk quantity on average. On the same instances with at least four valid model attempts, moving from one to four attempts increases the chance of finding a monitor false negative and erodes part of that reduction.”
“The deployment scheme is hypothetical; OpenAI does not deploy its current models in this way.”
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- Two new monitorability metrics, tested in simulations with known orderings and applied to past system card data for seven OpenAI agent models, with GPT-5 at high reasoning effort as monitor.
- What it found
- In simulation, pair matching recovers the known monitorability ordering where dataset-level g-mean² drifts. On system card data the two orderings generally agree; gating reduces risk but retries give some back.
- What it does not show
- Post-control retries are not adaptive, the scheme is hypothetical, and pair matching does not ensure models show equally hard-to-judge behavior. The post-control metric is not reported for intervention evaluations.
- Status
- The company’s own report about its own work
The record
- Questions
- Are AI companies putting stronger safeguards into practice? · Can AI systems act beyond what people intended or allowed?
- People
- Melody Guan (Wrote it) · Yuxiang Xie (Wrote it) · Bowen Baker (Wrote it)
- What we read
- We could read only part of the source.
- Found by
- The Radar, run of October 21, 2026, via
openai-alignment-blog(tier 1: the source’s own channel or an official record) - Cite this
- OpenAI Alignment Research Blog, “Beyond g-mean²: complementary metrics for monitorability”, October 20, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/openai-2026-10-20-monitorability-metrics-beyond-gmean