The BaseResearcharXiv

Tail-risk-aware Agent Safety: Constraining Tail Risks Across Multi-step LLM Agent Workflows

Researchers at the National University of Defense Technology propose TAS, a defence for LLM agents that treats a workflow as linked steps sharing one risk budget, aimed at rare but severe failures. They report lower attack success than eleven baselines on three benchmarks.

Open the source ↗3 quotations checked, word for word ↓

Why it matters

The same thing, for different readers.

What it means for you

Researchers describe a way to make AI agents that use tools less likely to be tricked into rare, serious mistakes, tested only on benchmark attacks so far.

For researchers

The authors argue that checking each agent step alone misses how risk builds across steps, and show that a shared budget on worst-case risk beats step-by-step guardrails on their benchmarks.

In their own words

Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.

From the source’s own text.

“By constraining tail risk rather than the overall average, TAS suppresses rare high-cost failures while preserving utility in normal cases.”
Shilong Pan, PDL Lab, National University of Defense TechnologyThe paper's abstract, stating what the proposed TAS framework is meant to achieve.
Found word for word in the source
“Since most agent behaviors are typically safe while a small fraction may incur severe failures, an overall-average risk measure can mask rare but severe risks.”
Shilong Pan, PDL Lab, National University of Defense TechnologyThe introduction, explaining why the authors measure worst-case rather than average risk.
Found word for word in the source
“Removing the shared budget constraint (w/o constraint) causes the largest safety degradation, showing that the budget is critical for controlling risk propagation.”
Shilong Pan, PDL Lab, National University of Defense TechnologyFrom the ablation study, where each component of TAS was removed in turn.
Found word for word in the source

How the finding was reached

In our words, from the source. Weigh the finding by how it was made.

What was done
Experiments on six LLM backbones (including Llama3-70B, Qwen2-72B, GPT-4o and GPT-5.2) using three agent benchmarks (Agent Security Bench, Agent-SafetyBench, AgentDojo), against eleven defence baselines, with ablations.
What it found
The authors report average safety improvements over No Defense of 47.95% and 70.25% under direct and indirect prompt-injection attacks, and a 58.69% reduction in attack success under backdoor attacks.
What it does not show
Benchmark attacks, scored by the authors' own cost scorer (rule-based plus an API-based semantic scorer). Tighter budgets block risky steps but may over-reject benign ones.
Status
A preprint: not yet peer reviewed

The record

Questions
Are AI companies putting stronger safeguards into practice? · Can AI systems act beyond what people intended or allowed?
People
Shilong Pan (Wrote it, PDL Lab, National University of Defense Technology) · Zhiliang Tian (Wrote it)
What we read
We read the whole source.
Found by
The Radar, run of November 8, 2026, via arxiv-ai-safety-query (tier 1: the source’s own channel or an official record)
Cite this
arXiv, “Tail-risk-aware Agent Safety: Constraining Tail Risks Across Multi-step LLM Agent Workflows”, November 7, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-11-07-tail-risk-aware-agent-safety
Our notesResults tables were hard to read as plain text, so the numbers in evidence come from the authors' prose. Utility trade-off and limits section (Appendix D) not read.