The BaseResearchLessWrong
LoReFT GRAFT Better at Preventing RL Grift
On LessWrong, Samuel Knoche reports that the GRAFT method against reward hacking can be improved by using a single-layer rank-1 LoReFT adapter as the forget adapter. The post says this uses 0.029% of the parameters of the original, suppresses hacking equally well, improves clean-environment solve rate and needs no anchor environment.
Why it matters
The same thing, for different readers.
For researchers
Samuel Knoche reports a far smaller forget adapter does the job of GRAFT's, though the post also discusses limits of fixed preventative steering baselines.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A variant of the GRAFT reward-hacking mitigation, replacing the forget adapter with a single-layer rank-1 LoReFT adapter.
- What it found
- The author reports equivalent reward hack suppression, using 0.029% of the parameters of the GRAFT paper's adapter, with a better clean-environment solve rate and no anchor environment.
- What it does not show
- Author-reported in a forum post and not independently reproduced; the post itself notes limits of fixed preventative steering baselines. Only the excerpt was read.
- Status
- A working paper: not yet peer reviewed
The record
- Questions
- Can AI systems act beyond what people intended or allowed? · Are AI companies putting stronger safeguards into practice?
- People
- Samuel Knoche (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of October 24, 2026, via
lesswrong(tier 1: the source’s own channel or an official record) - Cite this
- LessWrong, “LoReFT GRAFT Better at Preventing RL Grift”, October 24, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-10-24-loreft-graft-reward-hacking
Our notesLight lane: excerpt only. The relation is to the related gradient routing post, not GRAFT's own LessWrong post, which may not be in the Base; confirm or remove.