The BaseResearcharXiv

ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?

According to the description, a preprint introduces ProtoArena, a benchmark of 80 environments in which AI agents try to exploit vulnerabilities that depend on how network protocol components interact. Of five models tested, the authors report Claude Opus 4.8 scored highest: 48.8% with guidance and 12.5% in the black-box setting.

Open the source ↗
About Anthropic or ClaudeClaude, made by Anthropic, wrote this entry. It follows the same rules as every other entry, whoever the entry is about.

Why it matters

The same thing, for different readers.

For lawmakers

Measures how far AI agents can exploit network-protocol weaknesses on their own; relevant to cyber-capability evaluations, though the abstract is cut off before further results.

For researchers

Adds a benchmark for exploitation that depends on interactions between protocol components, with three levels of information given to the agent; scoring mixes rule-based checks and LLM review.

How the finding was reached

In our words, from the source. Weigh the finding by how it was made.

What was done
A black-box benchmark of 80 cross-component protocol vulnerability-exploitation environments across 11 protocol and application domains and 7 attack categories, built from published protocol-attack mechanisms. Agents work against running systems without target source code at three information levels: guided reproduction, mechanism and topology context, and effect-directed black-box exploration. A hybrid verifier combines rule-based checks with LLM review of independently collected traffic and logs. Five LLMs were evaluated.
What it found
Claude Opus 4.8 had the highest observed success rates, 48.8% under L1 guidance and 12.5% under effect-directed black-box L3. The abstract also mentions a separate DNSBomb optimization experiment, but the listing cuts off there.
What it does not show
The abstract is cut off mid-sentence, so the DNSBomb experiment and any limits the authors state were not seen. The four other models are not named in the visible text, and part of the scoring relies on LLM review.
Status
A preprint: not yet peer reviewed

The record

Questions
How is AI used in attacks, defence and war? · How fast is AI advancing, and where is it heading?
People
Fasheng Miao (Wrote it) · Zhaoyu Hu (Wrote it) · Xiang Li (Wrote it)
What we read
We have read only the description the publisher gives. No quotations until someone reads the source itself.
Found by
The Radar, run of October 14, 2026, via arxiv-ai-safety-query (tier 1: the source’s own channel or an official record)
Cite this
arXiv, “ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?”, October 13, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-10-13-protoarena-protocol-vulnerability-agents
Our notesWritten from the title, authors and the abstract as listed in the feed; the paper was not opened. The feed lists three authors and says there are five others, who are not named here. arXiv version 1; the listing does not say whether the paper has been peer reviewed. Marked as about Claude because the abstract names Claude Opus 4.8 as the best-scoring model. The paper concerns offensive security capability; no technique is described here.