The BaseResearchLessWrong
A big case study of propensity entanglement
According to the excerpt, nielsrolf links a paper on predicting how trained-in propensities generalize. The team elicited 24 propensities by four methods and tested spillover to other traits, finding that behavior of other models and system-prompting predict spillover best.
Why it matters
The same thing, for different readers.
For researchers
nielsrolf reports that fine-tuning a trait often spills over to other traits, and that simple predictors such as drift toward the base model work surprisingly well.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- 24 propensities elicited through 4 methods, with spillover to non-targeted traits compared across predictors; Qwen3.5-9B is mentioned.
- What it found
- Best predictor of SFT spillover is the behavior of two other models, then a system-prompted model, then drift.
- What it does not show
- Excerpt only; a SPAR project, link-post for the paper.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Can AI systems act beyond what people intended or allowed? · Are AI companies putting stronger safeguards into practice?
- People
- nielsrolf (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of October 27, 2026, via
lesswrong(tier 1: the source’s own channel or an official record) - Cite this
- LessWrong, “A big case study of propensity entanglement”, October 27, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-10-27-case-study-propensity-entanglement
Our notesLight lane: excerpt only; paper itself not read.