Aswin Surya
Titles as the sources gave them. In 1 entry, with 3 checked quotations.
What they said
Newest first. Each quotation was found word for word in the source.
“On average across probes, we find that the attack catch rate in simulation was only 1.0 points lower than in real runs, and the 95% interval (−3.0 to +1.0 points) includes 0.”
“We find that probes can distinguish simulated from real environments, but this doesn’t seem to affect their performance at detecting attacks.”
“Our evidence comes from just one agent, simulator, and suite of environments, but to our knowledge, this is the first study of its kind for probe monitors specifically.”
In the Base
Every entry they appear in.
Probe monitors seem to perform realistically in simulated control environments
Aswin Surya and James Lucassen, working at Redwood Research, test whether probes that read a model's internal activations catch attacks as well in simulated computer environments as in real ones. In one agent and one task suite, the catch rate differed little, though a probe could tell simulated from real.