The BaseResearcharXiv
Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing
According to the abstract, Andrew Trask, Sol Messing and colleagues propose double-blind evaluations using secure enclaves, so an evaluator's prompts stay hidden from the model owner while the model's weights stay hidden from the evaluator. They describe a pilot testing a Google DeepMind model against part of the private AILuminate benchmark.
Why it matters
The same thing, for different readers.
For lawmakers
Trask, Messing and colleagues describe a way regulators or auditors could test proprietary models without model owners seeing the test prompts or auditors seeing the weights.
For researchers
Trask, Messing and colleagues' pilot suggests private benchmarks could stay uncontaminated while model owners keep weights in their own infrastructure.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A pilot using an NVIDIA H100 secure enclave on Google Cloud Platform and OpenMined's PySyft to evaluate Gemini 2.5 Flash Lite against a selection of the private AILuminate benchmark corpus.
- What it found
- The authors say the pilot is the first use of this technology to give mutual secrecy for one organization's proprietary model weights and another's private prompts; the excerpt gives no safety results.
- What it does not show
- Not stated in the excerpt available.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Are AI companies putting stronger safeguards into practice? · Who can demand evidence, delay a release or stop an unsafe deployment?
- People
- Andrew Trask (Wrote it) · Sol Messing (Wrote it) · Vai Pahwa (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of November 10, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing”, November 9, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-11-09-double-blind-evals-safety-auditing