Jan Dubiński
Truthful AI; Warsaw University of Technology
Titles as the sources gave them. In 1 entry, with 5 checked quotations.
What they said
Newest first. Each quotation was found word for word in the source.
“Can SL transfer a wider range of traits, including more complex ones? If so, distillation might transfer subtle forms of misalignment (e.g., reward-seeking, scheming, and secret loyalties) without detection.”
“The student partially acquires the backdoor, responding in French on 23.5% of prompts with female names versus 0.0% with male names.”
“Finally, we test whether SL can transfer a propensity to hack in an agentic chess environment. We finetune the student on number sequences from a steered hacker teacher. The student hacks in 58.3% of episodes, compared with 10.9% for the unfinetuned model.”
“Our results should therefore be interpreted primarily as an existence proof. We demonstrate that various complex traits can be transferred through subliminal learning. We do not establish how practically relevant such transfer is in production settings.”
“Our findings raise the possibility that more complex forms of misalignment could be spread through model-generated training data, motivating further work on when such transfer occurs.”
In the Base
Every entry they appear in.
Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors
Using Qwen models, Truthful AI researchers tested whether a 'teacher' AI passes traits to a 'student' trained only on unrelated data, like number sequences. Students partly gained a new skill, a backdoor (French after female names on 23.5% of prompts) and a chess-hacking tendency (58.3% of episodes vs 10.9% before). The authors call it an existence proof in artificial settings.