MAS-Bench's shortcuts lift cross-app success from 17.0% to 61.7%; phoneshell's agent has one shortcut type of the three MAS-Bench tests

MAS-Bench scores mobile agents twice, once with GUI taps alone and once with access to 88 predefined shortcuts, APIs, deep links and RPA scripts, and the gap is largest on exactly the tasks that cross between apps. phoneshell's agent has a single shortcut verb, open_url, which covers one of MAS-Bench's three shortcut types and comes with no catalog to draw from.

MAS-Bench (arXiv:2509.06477), submitted 8 September 2025 by Pengxiang Zhao, Guangyi Liu and 12 coauthors from Zhejiang University and vivo AI Lab, and revised to its current version on 15 April 2026 ahead of ACL 2026, argues that GUI-only mobile agents are leaving an easy win on the table: real phones already carry structured ways to skip the screen, an API call, a deep link, a saved automation, and most agent benchmarks test as if those did not exist. The benchmark pairs 139 complex tasks across 11 real-world apps with a knowledge base of 88 predefined shortcuts, 11 APIs, 70 deep links, and 7 custom RPA scripts, and scores agents under both conditions: GUI-only, and hybrid, where the same agent can call a shortcut instead of tapping through a screen.

The gap is biggest on cross-app tasks, not single-app ones

Table 2 of the paper runs two agent families, MobileAgentV2 and T3A, both driven by Gemini-2.5-Pro, in both conditions, split into single-app tasks (92 of the 139) and cross-app tasks (47 of the 139). MobileAgentV2's GUI-only success rate on cross-app tasks is 17.0%; its hybrid version, MAS-MobileAgent, reaches 61.7%, a 44.7-point jump, the largest swing anywhere in the table. T3A's cross-app success rises from 34.0% GUI-only to 51.1% hybrid. On single-app tasks the hybrid advantage is smaller: MobileAgentV2 rises from 44.6% to 64.1%, T3A from 51.1% to 57.6%. The paper's own summary calls the hybrid single-app result "a significant improvement over the 44.6% achieved by their GUI-only counterparts," and a second model pairing, Gemini-2.0-Flash (Table 5), shows the same direction: MobileAgentV2's cross-app success rises from 0% GUI-only to 23.4% hybrid.

Predefined shortcuts beat agent-generated ones, and phoneshell has neither

MAS-Bench does not stop at predefined shortcuts. Table 3 also tests whether an agent can discover or build its own: replaying a prior successful task, replaying a subtask, generating a shortcut dynamically, or using a dedicated shortcut-authoring agent (MobileAgent-E). Predefined shortcuts reach a 1.00 shortcut success rate (SSR), meaning every attempted use of one actually works, at an overall task success rate of 0.52 against a no-shortcut baseline of 0.43. Agent-generated shortcuts lag: Replay-Task shortcuts manage only a 0.10 SSR and drop task success to 0.34, below the no-shortcut baseline. The paper's own framing is "agent-generated shortcuts lag in robustness and efficiency" compared to a curated knowledge base.

phoneshell's (github.com/Blomega/phoneshell) action surface, read from phoneshell/actions.py and the tool list exposed in phoneshell/mcp_server.py, has exactly one verb that matches any of MAS-Bench's three shortcut categories: open_url, a deep-link opener. Its own docstring states the case for it plainly: "Deep links are the fastest, least fragile way into a known screen." There is no equivalent for an API call (phoneshell drives the phone through WebDriverAgent and the accessibility tree, not through any app's backend) and no equivalent for an RPA script (there is no saved, replayable action sequence an agent can invoke by name; phoneshell/agent/macros.py records and replays a fixed macro by a human, not a shortcut an agent chooses mid-task). There is also no knowledge base: no file in the repository catalogs known-good deep links the way MAS-Bench's 70-deep-link set does, so an agent has to already know or guess a URL scheme before open_url is useful at all.

What it means for anyone building a phone agent's action space

• The cross-app result is the sharpest evidence in the paper: a 44.7-point swing from a single added capability suggests that GUI-only agents are not failing because they are bad at tapping, they are failing because moving between apps through the UI alone is disproportionately expensive and error-prone, and a shortcut removes that cost entirely rather than reducing it.
• phoneshell's single shortcut, open_url, already covers the category MAS-Bench's own data calls the cheapest to add and easiest to get right (deep links dominate the 88-entry knowledge base at 70 of 88). It is also the category phoneshell's own code never catalogs, so the benefit is theoretically available but not actually realized without an agent that already knows the right scheme.
• The predefined-versus-generated gap (1.00 SSR against 0.10) is a caution against the obvious next step. Letting phoneshell's agent write its own shortcuts, rather than using a maintained list, is the MAS-Bench condition that performed worst of all five tested, worse than no shortcuts at all.

Check it yourself

# phoneshell's only shortcut-shaped action
grep -n "def open_url" phoneshell/actions.py
grep -n "phone_open_url\|open_url" phoneshell/mcp_server.py

# confirm there is no API-call action and no agent-invokable script catalog
grep -rn "def \|open_url\|shortcut" phoneshell/actions.py | grep -v "^Binary"
find . -iname "*shortcut*" -not -path "./site/*"

# MAS-Bench's own tables and shortcut knowledge base
curl -s https://arxiv.org/abs/2509.06477 | grep -i "shortcut\|deep-link\|RPA"
# full tables: https://arxiv.org/html/2509.06477v1 (Table 2, Table 3, Table 5)

What would prove this wrong

We claim phoneshell's action space contains exactly one shortcut type, a deep-link opener, and no API-call path, no agent-invokable automation script, and no catalog of known-good links. That is wrong the moment a future commit to phoneshell/actions.py or phoneshell/mcp_server.py adds an action that calls an app's API directly, lets the agent invoke a saved macro by name mid-task, or ships a file cataloging deep links the way MAS-Bench's 70-entry set does. We also report MAS-Bench's cross-app gap as 44.7 points from its own Table 2; that number is wrong if a later revision of the paper restates Table 2 with different figures. We will check both by 31 March 2027.

Sources

• Zhao, Liu, et al., MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI Agents, arXiv:2509.06477, submitted 8 September 2025, revised 15 April 2026 (v2), read 17 October 2026.
• Full tables (Table 2 GUI-only vs hybrid, Table 3 predefined vs agent-generated shortcuts, Table 5 Gemini-2.0-Flash): arxiv.org/html/2509.06477v1.
• Blolabel, phoneshell: phoneshell/actions.py, phoneshell/mcp_server.py, phoneshell/agent/macros.py, read 17 October 2026.