D-GARA's permission dialogs and battery warnings cut AgentCPM-GUI-8B from 59.87% to 26.97%; phoneshell detects the same alert and never scores what it cost

D-GARA (arXiv:2511.16590, accepted AAAI 2026) interrupts five GUI agents mid-task with the anomalies a real phone actually produces: low-battery warnings, permission dialogs, app crashes, update prompts. The worst of the five models loses 32.9 points. phoneshell's own code already detects the same kind of system alert on a real iPhone; its results schema has no field that would let anyone compute whether a phoneshell task failed because of one.

D-GARA (arXiv:2511.16590, submitted 20 November 2025 by Sen Chen, Tong Zhao, Yi Bin, Fei Ma, Wenqi Shao and Zheng Wang, accepted to AAAI 2026) ran five GUI agents across roughly 150 Android tasks twice: once clean, once with a real-world interruption injected mid-task. AgentCPM-GUI-8B, the weakest of the five on this measure, dropped from 59.87% to 26.97%, a 32.90-point fall that leaves it completing fewer than four in ten of the tasks it completed without the interruption (a relative success rate of 39.56%). Every one of the five models tested lost ground. None of the interruptions were adversarial or exotic: a low-battery banner, a Wi-Fi drop, an app update prompt, a permission dialog, a crash. phoneshell's own code already has a name for one of those, Alert, and already reads its text and buttons off a real iPhone. Nothing in phoneshell's results schema records whether one appeared.

What D-GARA injects, and what it cost each model

D-GARA groups its interruptions into five categories: System Resource (low-battery warnings, thermal throttling alerts), System Network (Wi-Fi disconnection, mobile data switches), App Malfunction (crashes, freezes), Permission Control (runtime permission dialogs), and UX Disruption (update prompts, feedback forms). The benchmark runs 152 tasks built around apps including JD.com, Weibo, Bilibili, Amap, Ctrip, Amazon, Facebook and Google Maps, per the paper's Appendix A, though a separate table in the same paper, labelled D-GARA-152, lists seven apps for what appears to be the same split, a discrepancy the paper does not resolve and we could not verify further. Five models were evaluated with and without the anomaly injection turned on:

Source: D-GARA, arXiv:2511.16590, Table 1, read 27 October 2026. Relative success rate is success-with-interruption divided by success-without.

A separate, more severe probe in the same paper forces agents down a harder interaction path, a single confirm button standing in for what a two-button dialog would normally offer: Qwen2.5-VL-7B falls from 96.15% to 41.27%, and AgentCPM-GUI-8B falls from 82.35% to 9.30%. The paper's own framing is that none of these five models hold up against anomalies a real device routinely produces on its own, with no attacker involved.

phoneshell's own alert handling, and the hole next to it

phoneshell already detects exactly one of D-GARA's five categories at the perception layer. actions.py's snapshot routine checks whether any element on screen has type Alert or Dialog; if one does, it asks WebDriverAgent for the alert's text and buttons, a call the code's own comment says costs 1.36 seconds and is worth paying only when the tree already flagged an alert as present. That text reaches the model: observation.py's as_text() method writes a line reading "!! SYSTEM ALERT ON SCREEN" straight into what the model sees, and actions.py exposes accept_alert and dismiss_alert as callable actions, with an MCP tool, phone_alert, wired into the Claude Code CLI path in bench/runner.py. None of the other four categories, no network state, no battery level, no update prompt, no crash detector, has any equivalent anywhere in the codebase.

What is missing sits one layer up. bench/runner.py's TaskResult dataclass, the record written to results.json for every run, carries task_id, model, passed, checks, turns, cost_usd, seconds, agent_said and error. There is no field for whether a system alert appeared during the task. phoneshell can see the exact anomaly D-GARA calls Permission Control, hand the model the means to deal with it, and then throw away, at the moment of scoring, the one piece of information that would say whether that interruption had anything to do with the outcome. A task that fails because a model mishandled a real alert on a real iPhone is written to the same JSON row, with the same passed: false, as a task that fails for any other reason.

What it means for a benchmark run on one real phone

• D-GARA's number is a controlled comparison: the same tasks, with and without the injection, on the same models. phoneshell runs on a single physical iPhone, where alerts show up or not depending on the phone's own state (notification settings, first-launch permission prompts, the carrier's own banners), not on a deliberate toggle. The two are not directly comparable, but the direction D-GARA reports, that an ordinary system interruption costs real points, is exactly the kind of effect phoneshell's current data cannot rule out or confirm.
• phoneshell's detection code already distinguishes a system_alert from an ordinary tap target in at least one place (actions.py's consistency check compares against found.kind != "system_alert"), so the information exists in memory during a run. It is simply never written down.
• The fix implied by D-GARA's result is not a new detector. phoneshell already has the detector. It is a one-field addition to TaskResult, logging whether snap.alert was non-null at any point during the task, which would let a future run compute phoneshell's own relative success rate the way D-GARA computes its.

Check it yourself

# phoneshell's alert detection, all of it
grep -n "e.type in {\"Alert\", \"Dialog\"}" -A5 phoneshell/actions.py
grep -n "SYSTEM ALERT ON SCREEN" phoneshell/agent/observation.py
grep -n "phone_alert\|accept_alert\|dismiss_alert" phoneshell/actions.py phoneshell/bench/runner.py

# confirm TaskResult has no field for whether an alert occurred
grep -n "class TaskResult" -A16 phoneshell/bench/runner.py

# D-GARA's own anomaly categories and Table 1
curl -s https://arxiv.org/html/2511.16590v1 | grep -iE "permission|battery|Table 1"

What would prove this wrong

We claim phoneshell detects system alerts at the perception layer but records nothing about them in its results schema, so no one can currently compute whether a phoneshell task failure correlates with one appearing. That is wrong if a later version of bench/runner.py's TaskResult carries an alert-related field we missed, or a build we did not check. We also claim D-GARA's degradation, measured on Android apps, has not been shown to hold on phoneshell's own iOS task set. That is tested directly the day someone adds the one field above and re-runs phoneshell's existing 80 tasks enough times to see whether the runs that happened to trigger a real alert score worse than the ones that did not. We will treat that comparison, if published, as the relevant check, by 30 April 2027.