The BasePeople

Jeffrey Olmo

Titles as the sources gave them. In 1 entry, with 4 checked quotations.

What they said

Newest first. Each quotation was found word for word in the source.

“We found that the gap between simulated and real was often about as small as the gap between two independent real runs.”
Simulated Control EvaluationsRedwood Research, October 23, 2026. Speaking as: Other. The authors summarise a check of their method against real execution in LinuxArena environments.
“Its error in predicting the main safety number is 3.6 percentage points, versus 10.1 for BashArena.”
Simulated Control EvaluationsRedwood Research, October 23, 2026. Speaking as: Other. The authors summarise the result for simulation at a 5% audit budget, compared with the simpler BashArena setting.
“That said, we can only verify simulator fidelity in settings where real execution is possible, so applying SCEs to deployment involves some extrapolation.”
Simulated Control EvaluationsRedwood Research, October 23, 2026. Speaking as: Other. The authors state a main limit of the method after reporting positive results.
“The evidence is now strong enough that we’d be excited to see AI developers running them.”
Simulated Control EvaluationsRedwood Research, October 23, 2026. Speaking as: Other. The authors say what they hope AI developers will do with simulated control evaluations.

In the Base

Every entry they appear in.

  1. ResearchRedwood ResearchWrote it

    Simulated Control Evaluations

    Redwood Research authors Jeffrey Olmo and Tyler describe a paper proposing simulated control evaluations, in which a language model fakes the tool outputs so attacks on AI agent monitors can be tested without executing them. They report that results track real execution closely in LinuxArena, and list limits.