GhostEI-Bench's worst model completes 56.25% of the injected-overlay attacks it should refuse, GPT-5 only 16.43%; phoneshell's own overlay reflex blocks the same words and hands the rest to the model anyway

GhostEI-Bench injects fake overlays and spoofed notifications into an Android agent's own screen and measures how often that gets an attacker's goal partly or fully completed: a Vulnerability Rate of 16.43% for the best model tested, GPT-5, and 56.25% for the worst, UI-TARS-7B-SFT. phoneshell's harness-level reflex for clearing overlays refuses, by construction, to tap any control whose text contains pay, confirm, allow, send or ten other words. When that is the only way out, it tries three gestures that never read the overlay's content at all, then hands the whole thing to the model unresolved, which is the exact decision GhostEI-Bench's numbers say models get wrong.

GhostEI-Bench (arXiv:2510.20333, Chiyu Chen, Xinhao Song, Yunkai Chai, Yang Yao, Haodong Zhao, Lijun Li, Jie Li, Yan Teng, Gongshen Liu and Yingchun Wang, submitted 23 October 2025, revised to v3 on 5 March 2026, accepted as an ICLR 2026 poster) names a threat it calls environmental injection: a fake overlay or a spoofed notification placed into the screen an agent sees, built to bypass text-based safety filters because the attack lives in the pixels, not the instruction. Running Mobile-Agent-v2 and AppAgent against injected attacks across seven critical-risk categories, the paper's Table 1 reports a Vulnerability Rate, the share of non-benign-failure trajectories that fully or partially completed the attacker's goal, from 16.43% for GPT-5 up to 56.25% for UI-TARS-7B-SFT. phoneshell's own reflex for clearing exactly this class of screen element, a sheet or popover in the way of the actual task, was built for a different reason entirely, getting past promotional interstitials cheaply, and it already refuses by construction to tap the words these attacks would need tapped.

What GhostEI-Bench measures, and where its numbers land

GhostEI-Bench's attack model is specific: it is not a prompt injection in the text the agent reads, it is a visual one, a fake "Update Required" sheet, a spoofed payment confirmation, a deceptive notification, rendered into the same screen the agent is already looking at inside a real app workflow on an Android emulator, across seven critical-risk fields (Fraud, Cybercrime, Disinformation, System Sabotage, Privacy Leakage, Copyright Infringement, Harassment) and seven application domains (Communication, Finance, Social Media, Web Navigation, Productivity, Settings, Life Services). A judge LLM reviews the full trajectory, screenshots and actions together, and labels each run Task Completion, Full Attack Success, Partial Attack Success or Benign Failure; Vulnerability Rate is (FAS + PAS) divided by every case that was not a benign failure. Table 1 reports results for seven models under Mobile-Agent-v2 and two of those seven repeated under AppAgent, plus two UI-TARS variants whose framework the paper's text does not name, a gap we are stating rather than guessing past.

Source: arXiv:2510.20333, Table 1, read 28 October 2026. The paper's Section 4.2 text calls Claude-3.7-Sonnet's 55.12% "the highest of all models," which conflicts with Table 1's own 56.25% for UI-TARS-7B-SFT; we report both numbers as the table states them rather than resolving the discrepancy ourselves.

What phoneshell's overlay reflex already refuses, and what it does instead

phoneshell/agent/overlays.py was not built for attacks. Its own docstring states the problem it solves: "the single most common way a phone agent stalls" is a promotional sheet covering the screen, and dismissing it is "a reflex with maybe six known moves" that belongs in the harness, not the model, because it costs nothing and cannot loop. Three boundaries are stated explicitly in the same docstring: permission dialogs are never auto-answered, anything that reads as money, sending or deleting is never tapped, and a dismissal must be verified gone rather than assumed. The second boundary is enforced by one regular expression, NEVER, checked inside _is_close_control against every candidate's label, name and value: pay, buy, order, checkout, purchase, subscribe, confirm, send, delete, remove, allow, don't allow, sign out, log out, agree, accept. Any element whose text matches is excluded from the reflex's list of safe things to tap, full stop, whether or not it looks like the overlay's intended close control.

That exclusion does not make the reflex give up. dismiss() runs through five further moves in order, and three of them never read the overlay's content at all: dragging the sheet down from its grabber or top edge, tapping the dimmed backdrop above it, and swiping back from the screen edge. Each of these can close a sheet whose only labelled button says "Confirm" or "Allow" without ever tapping that button, because the gesture targets geometry, not text. Only when drag, backdrop and back-swipe all fail does dismiss() return an unresolved result, carrying the overlay's own text and a note on everything that was tried, straight back into the model's context. A true system alert, Apple's own permission or confirmation dialog, skips all of this and is refused immediately, on the stated principle that answering it is the user's call. GhostEI-Bench's injected elements are fake overlays rendered inside an app's own view, not a genuine system alert, so they would be classified as a sheet or fullscreen modal and run the full five-step gauntlet before ever reaching a model, if they reached phoneshell at all.

What it means for a harness that was not built against attacks

• phoneshell's word blocklist and GhostEI-Bench's attack vocabulary overlap by design accident, not by any shared threat model. NEVER exists to stop the reflex from tapping "Subscribe" on a benign paywall; that it also covers "Confirm" and "Allow" is a side effect of trying to list every word that reads as irreversible, not a response to environmental injection.
• The gesture fallback is content-blind in both directions. It can dismiss a malicious overlay without ever understanding it was malicious, which GhostEI-Bench's own category list would count as a benign-looking escape rather than a judged defense. It can just as easily drag away, or swipe back from, a legitimate screen that happens to share a sheet's shape, with no distinction between the two anywhere in the code.
• Nothing in phoneshell's own test suite exercises this path. A grep of environments/*.yaml for any task that plants a deceptive or spoofed overlay mid-task returns nothing; every overlay phoneshell's 80 tasks have ever met is incidental, a real app's own promo sheet, not a planted adversarial one.

Check it yourself

# phoneshell's word blocklist, verbatim
grep -n "NEVER = re.compile" -A4 phoneshell/agent/overlays.py

# the exclusion runs inside the close-control check, before any gesture
grep -n "_is_close_control\|NEVER.search" phoneshell/agent/overlays.py

# the three content-blind fallbacks, in order, inside dismiss()
grep -n "drag the sheet down\|tap the dimmed backdrop\|back edge swipe" -B2 phoneshell/agent/overlays.py

# system alerts are refused outright, before any of the above runs
grep -n "system_alert" -A4 phoneshell/agent/overlays.py

# GhostEI-Bench's own Table 1 and Vulnerability Rate definition
curl -s https://arxiv.org/html/2510.20333v3 | grep -i "Vulnerability Rate\|VR ="

What would prove this wrong

We claim phoneshell's overlay-dismissal reflex has never been tested against anything resembling GhostEI-Bench's injected adversarial overlays, so whether its word blocklist plus gesture fallback performs better, worse, or incomparably to the Android agents in Table 1 is unknown. That is wrong if a GhostEI-Bench-style injected overlay has already been run against phoneshell's dismiss() on a real iPhone and we did not find the record. We also claim the gesture fallback cannot distinguish a malicious overlay from a legitimate one that shares its shape. That is falsified the day someone builds one phoneshell task with a planted deceptive sheet whose only path forward is a NEVER-listed word, and the reflex's drag, backdrop and back-swipe sequence reliably clears it without the model ever seeing the deception, which would mean the content-blind fallback is doing more real defensive work than this piece credits it for. We will treat either event, published or reproduced, as grounds to revisit this comparison by 30 April 2027.