The BaseResearcharXiv
Soei: Structured Observability of Evidence and Intent for Multi-Agent AI Software Engineering
According to the abstract, Mohammed Nayeem Khan and K S Sarun Kumar propose Soei, a system for tracking what AI coding agents actually did. On 45 real sessions they estimate 23.4% of code-editing episodes end with no independent check and 57.8% of completed to-dos lacked proof.
Why it matters
The same thing, for different readers.
What it means for you
Mohammed Nayeem Khan and K S Sarun Kumar estimate that AI coding agents often mark work done with no independent check, so agent-built software may be less verified.
For researchers
The authors report estimates from 45 sessions using a blind-audited classifier, and an audit of an open-source agent observability tool that found an unauthenticated local mutation surface.
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A prototype run on 45 real coding-agent sessions (102,233 records, 20,577 tool executions, 2,336 human-to-agent episodes), plus a source-level verification of the open-source tool Agenttrail.
- What it found
- The authors estimate 23.4% (95% CI 17.1 to 35.7) of code-editing episodes end with no independent check after the last edit, and 57.8% of agent-completed to-dos had no proof in between; 42% of sessions run longer than 24 hours.
- What it does not show
- Small sample of 45 sessions; classifier precision reported as 83.3% and 96.7% on two audits. Excerpt truncated.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Are AI companies putting stronger safeguards into practice? · Can AI systems act beyond what people intended or allowed?
- People
- Mohammed Nayeem Khan (Wrote it) · K S Sarun Kumar (Wrote it)
- What we read
- We have read only the description the publisher gives. No quotations until someone reads the source itself.
- Found by
- The Radar, run of October 30, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “Soei: Structured Observability of Evidence and Intent for Multi-Agent AI Software Engineering”, October 29, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-10-29-soei-coding-agent-observability