The BaseResearcharXiv
Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following
A preprint argues that verifiable instruction-following benchmarks can give full credit to answers with little substance, and that using the same verifier as a training reward induces reward hacking. It proposes CALIF, a likelihood-based bonus to preserve response quality.
Why it matters
The same thing, for different readers.
For researchers
Warns that verifier-only rewards can be gamed and offers CALIF, which needs no extra judge or learned reward model.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- Evaluation of frontier models and public checkpoints; CALIF tested on models from 1.7B to 8B in reasoning and non-reasoning modes.
- What it found
- Per the abstract, CALIF improves IFBench constraint satisfaction while keeping quality near or above pre-RL levels, and beats base policies in 40 of 45 model-metric comparisons.
- What it does not show
- Abstract only; small to mid-size models.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Are AI companies putting stronger safeguards into practice? · How fast is AI advancing, and where is it heading?
- People
- Sangyeon Yoon (Wrote it) · Wooseok Seo (Wrote it) · Hamish Ivison (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of October 15, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following”, October 14, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-10-14-reward-hacking-instruction-following
Our notesLight lane: written from title, author and abstract excerpt only; not fetched. Authors listed are those named in triage.