The BaseResearcharXiv
SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents
Researchers at the National University of Singapore and partner institutions introduce SWE-Misbehavior, a 7.4K-case benchmark of coding agents acting outside their task. Testing four agent frameworks, they report violation rates up to 98.5% on one check, and a tool-restriction method, Code-KIT, that cut average violations by 52% for one agent and 32% for another.
Why it matters
The same thing, for different readers.
What it means for you
In these tests, AI coding helpers often read, created or changed things nobody asked about, such as extra files and software installs, even when the main job looked finished.
For lawmakers
Judging coding agents on whether the task succeeded misses side effects; the authors call for checking that work stays within bounds users expect, with human oversight in production.
For researchers
The authors tie rates to the agent framework more than the model; restricting tools beat prompt rules, though AI-judge agreement was weak on some metrics and Code-KIT can be bypassed.
The question it raises
When an agent does more than it was asked but nothing visibly breaks, who is responsible: the user who granted broad access, or the tool that left write access open?
In their own words
Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.
From the source’s own text.
“Experiments on state-of-the-art open-source coding agents (e.g., OpenHands, smolagents) reveal substantial misaligned behaviors, including dependency misinterpretation (up to 98.5% of cases), unintended modification of user inputs (50.5%), and silent compromise of environment integrity (53%).”
Found word for word in the source
“The same models behave very differently depending on the agent framework, which shows that the tool interface plays a key role.”
Found word for word in the source
“This verifies that prompting alone changes the agent’s awareness but not its affordances; because agents still rely on overly broad general-purpose tools, misbehaviors like Context and Body Change actually worsen in OpenHands.”
Found word for word in the source
“Motivated by these findings, we further propose Code-KIT, a lightweight adaptive framework that combines misbehavior-aware guidelines with novel tools to help agents avoid such failures. Code-KIT reduces misbehaviors by 52% for smolagents and 32% for OpenHands, while simultaneously improving task completion performance.”
Found word for word in the source
“Although SWE-Misbehavior makes solid progress in studying misbehaviors of coding agents, it does not cover the full spectrum of failures that may arise in agentic environments, such as multi-turn security vulnerability injection or complex unauthorized data exfiltration.”
Found word for word in the source
How the finding was reached
In our words, from the source. Weigh the finding by how it was made.
- What was done
- A benchmark of 7.4K cases built from 115 real GitHub repositories (function-completion tasks from the DevEval benchmark), covering code generation, file management (delete, add, move) and system interaction (execute, test), with 11 automatic checks for boundary violations such as changing unrelated files, creating unrequested files, altering the environment or not meeting the task goal. Four agent frameworks (OpenHands, smolagents, Agno, Claude Code) were tested with GPT-5, Gemini-2.5-Flash and Qwen3-Coder-30B (Claude Code with Claude Sonnet 4.5) on a shared sample: 200 cases per code-generation level, 200 file-management and 200 system-interaction cases. Four checks are scored by an AI judge (Gemini-3-Flash) whose labels were compared with three human annotators on 200 sampled runs. The mitigation, Code-KIT, adds guidelines and five purpose-built tools chosen per task by an AI selector; it was tested on smolagents and OpenHands with Gemini-2.5-Flash (Qwen3-Coder-30B in the appendix).
- What it found
- Violations were common while tasks often still appeared to complete. The File Read check (not inspecting required files, or reading beyond the required set) flagged up to 98.50% of runs on system-interaction tasks and 21.67% to 33.33% on code generation. Environment changes beyond the task reached 53.00% (OpenHands with Gemini-2.5-Flash, system interaction). Unrequested changes to the given code or docstrings reached up to 59.33% in code generation (the abstract says 50.5%), and unrequested test or draft files appeared in up to 79% of system-interaction runs. The authors tie the differences to the agent framework and its tools more than to the model. Adding rules to the prompt gave inconsistent results. Code-KIT cut the average misbehavior rate with Gemini-2.5-Flash from 12.77% to 6.11% (52%) for smolagents and from 18.19% to 12.31% (32%) for OpenHands; runs judged not to meet the task goal fell from 32.11% to 23.89% and from 40.06% to 24.78%.
- What it does not show
- The authors say the benchmark does not cover failures such as multi-turn vulnerability injection or unauthorized data exfiltration, that Code-KIT will need different tool designs for longer, open-ended work, and that agents can bypass its tools through native editors or shell commands. Judge agreement with humans on 200 sampled runs was 96.5% for File Read, 72.0% for Instruction Following, 92.5% for Environment Integrity and 93.5% for Resource Exhaustion, but chance-corrected agreement (Cohen's kappa) was 0.85, 0.37, -0.01 and -0.02; the authors attribute the near-zero values to rare positives in two of the three task families. By design, the File Read check also counts reading more than a set list of files, and all checks count departures from the task whether or not they cause harm. Code-KIT was tested on two of the four frameworks and mainly one model. Results come from sampled cases built from a single source benchmark.
- Status
- A preprint: not yet peer reviewed
The record
- Questions
- Can AI systems act beyond what people intended or allowed? · Are AI companies putting stronger safeguards into practice?
- People
- Do Xuan Trong (Wrote it, National University of Singapore; Vietnam National University, Hanoi) · Duc Anh Nguyen (Wrote it, National University of Singapore; University of Virginia, USA) · Do Xuan Long (Wrote it, National University of Singapore; Agency for Science, Technology and Research (A*STAR), Singapore) · Ngoc-Hai Nguyen (Wrote it, National University of Singapore; Tufts University, USA) · Tongyao Zhu (Wrote it, National University of Singapore) · Duy Dinh (Wrote it, National University of Singapore) · Cayden Chik (Wrote it, National University of Singapore) · Cuong Dang (Wrote it, Virginia Tech, USA) · Jiaying Wu (Wrote it, National University of Singapore) · Xuan-Phi Nguyen (Wrote it, Salesforce AI Research, USA) · Vinh Van Nguyen (Wrote it, Vietnam National University, Hanoi) · Kenji Kawaguchi (Wrote it, National University of Singapore) · Shafiq Joty (Wrote it, Salesforce AI Research, USA) · Nancy F. Chen (Wrote it, Agency for Science, Technology and Research (A*STAR), Singapore) · Min-Yen Kan (Wrote it, National University of Singapore)
- What we read
- We read the whole source.
- Found by
- The Radar, run of October 14, 2026, via
arxiv-ai-safety-query(tier 1: the source’s own channel or an official record) - Cite this
- arXiv, “SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents”, October 13, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/arxiv-2026-10-13-swe-misbehavior-coding-agent-boundary-violations
Our notesPreprint, not peer reviewed (arXiv v1, 13 Oct 2026). Full text read from the arXiv HTML version, not the PDF, so no lost-ligature problem. Quotes are co-authored paper text; speaker set to the first author per the run rule (Do Xuan Trong and Duc Anh Nguyen are marked equal contribution; Do Xuan Long is project lead). Check: (1) the arXiv listing abstract in triage says Code-KIT cuts misbehavior 'by 34% for OpenHands'; the paper's own abstract, text and Table 4 say 32% (18.19 to 12.31), which we use. (2) The abstract gives 50.5% for unintended modification of user inputs, but Table 2 and Finding 2 give up to 59.33% (OpenHands, GPT-5, code generation; 50.50% is smolagents, GPT-5). (3) The 98.5% figure is the File Read check, which flags too few or too many files read. (4) Cohen's kappa for the Environment Integrity judge, the check behind the 53% figure, is about zero (-0.01); the paper does not say how many of the 200 sampled runs were system-interaction runs. (5) The testing prompt allows an isolated test harness, yet File Create flags unrequested files left behind; the 79% figure (Claude Code row, Table 2) may partly reflect that, our reading, not discussed by the authors. Claude Code (with Claude Sonnet 4.5) is one of four frameworks tested and Claude Sonnet 4.5 was one of three candidate judges; neither is the paper's subject, so conflict is false and anthropic is left out of orgs. Table 2 averages for Claude Code: 3.80% (code generation), 8.50% (file management), 18.95% (system interaction); the paper does not discuss them separately. Several authors list two affiliations; titles show them as given.