Verifying Agents in Rubric-Graded Environments

According to the abstract, Markus Dücker and colleagues study how to verify AI agents' work against written rubrics. They build a dataset of 2,464 human-judged criteria from investment-banking tasks and release an open-source verifier, Gandalf, which they report matches or beats three baselines at lower cost.

Why it matters

The same thing, for different readers.

For researchers

The authors report the first systematic study of verifiers for rubric-graded agent tasks, with a nine-capability taxonomy and a verifier matching the strongest baseline at one-fifth of the cost.

How the finding was reached

In our words, from the source. Weigh the finding by how it was made.

What was done
BankerVerifierBench: 2,464 human-judged rubric criteria across 24 investment-banking tasks; the Gandalf verifier compared with three existing verifiers.
What it found
With Gemini 3 Flash, Gandalf matches the strongest baseline's F1 (0.864 versus 0.852) at one-fifth of its cost; with GPT-5.4 models it improves F1 by over 0.12 at lower cost, as the authors report.
What it does not show
Read from the abstract excerpt only; results come from one domain, investment-banking tasks.
Status
A preprint: not yet peer reviewed

The record

Questions
Are AI companies putting stronger safeguards into practice?
People
Markus Dücker (Wrote it)
What we read
We have read only the description the publisher gives. No quotations until someone reads the source itself.
Found by
The Radar, run of November 18, 2026, via arxiv-ai-safety-query (tier 1: the source’s own channel or an official record)
Cite this
arXiv, “Verifying Agents in Rubric-Graded Environments”, November 17, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-11-17-verifying-agents-rubric-environments
Our notesLight lane: written from the triage excerpt only; nothing fetched and no source text saved.