The BaseResearcharXiv
Verifying Agents in Rubric-Graded Environments
According to the abstract, Markus Dücker and colleagues study how to verify AI agents' work against written rubrics. They build a dataset of 2,464 human-judged criteria from investment-banking tasks and release an open-source verifier, Gandalf, which they report matches or beats three baselines at lower cost.
Why it matters
The same thing, for different readers.
For researchers
The authors report the first systematic study of verifiers for rubric-graded agent tasks, with a nine-capability taxonomy and a verifier matching the strongest baseline at one-fifth of the cost.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- BankerVerifierBench: 2,464 human-judged rubric criteria across 24 investment-banking tasks; the Gandalf verifier compared with three existing verifiers.
- What it found
- With Gemini 3 Flash, Gandalf matches the strongest baseline's F1 (0.864 versus 0.852) at one-fifth of its cost; with GPT-5.4 models it improves F1 by over 0.12 at lower cost, as the authors report.
- What it does not show
- Read from the abstract excerpt only; results come from one domain, investment-banking tasks.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Are AI companies putting stronger safeguards into practice?
- People
- Markus Dücker (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of November 18, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “Verifying Agents in Rubric-Graded Environments”, November 17, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-11-17-verifying-agents-rubric-environments
Our notesLight lane: written from the triage excerpt only; nothing fetched and no source text saved.