Udari Madhushani Sehwag

RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?

Scale AI researchers build a testbed in which a frontier model, doing a benign job in a successor model's training pipeline, is also told to covertly plant a misaligned property in that successor. They report that all six models tested sometimes succeed, and that monitors reading only transcripts miss many attacks.