UI-CUBE's best model falls from 84.8% on simple UI tasks to 19.4% on its 50 copy-paste workflows; phoneshell has a working three-finger copy and paste gesture that zero of its 80 tasks ever call
UI-CUBE splits 226 enterprise computer-use tasks into simple UI interactions and complex workflows built around copying data between applications. Its five tested agents score 66.7% to 84.8% on the simple tasks and collapse to 9.5% to 19.4% on the complex ones, against a human baseline of 97.9% and 61.2%. phoneshell/gestures.py already implements the exact iOS move, a three-finger pinch, that performs copy and paste on a real phone. No task in its public suite has ever asked an agent to use it.
UI-CUBE (arXiv:2511.17131, Horia Cristescu, Charles Park, Trong Canh Nguyen, Sergiu Talmacel, Alexandru-Gabriel Ilie and Stefan Adam, submitted 21 November 2025, benchmark and code published under UiPath's GitHub organization) argues that task accuracy alone understates how far current computer-use agents are from enterprise deployment, and backs that with a specific split: 136 simple UI-interaction tasks, 50 copy-paste workflows and 40 enterprise application scenarios, 226 in total. Averaged across resolutions in the paper's Table 5, five agents score between 66.7% and 84.8% on simple tasks and between 9.5% and 19.4% on complex ones, a drop the paper attributes to architectural limits in memory management, hierarchical planning and state coordination rather than to anything a bigger model fixes on its own. phoneshell has never run anything like UI-CUBE's copy-paste category, but not for lack of a gesture: the move exists in its code, unused.
The collapse, and where it comes from
UI-CUBE evaluates five agents, Claude Computer Use 4.0, OpenAI-computer-use-preview, and three configurations of UiPath's own ScreenAgent (on Gemini 2.5 Flash, GPT-5 mini and GPT-5), scored against application state rather than an LLM judge, across multiple screen resolutions. Table 5's averages show every one of the five dropping by at least 47 points moving from simple tasks to complex ones, and the paper's human baseline, Table 8, drops too: 97.9% on simple tasks to 61.2% on complex ones, people with no prior experience in the specific application. The gap between the best model's complex score, 19.4%, and the weakest human application score the paper reports, 50.0% on Concur and SAP Stock in Table 7, is itself larger than the best model's entire simple-task score minus its complex one for three of the five agents.
| Agent | Simple tasks | Complex tasks | Drop |
|---|---|---|---|
| Claude Computer Use 4.0 | 66.7% | 9.5% | −57.2 pts |
| OpenAI-computer-use-preview | 70.3% | 10.5% | −59.8 pts |
| UiPath ScreenAgent / Gemini 2.5 Flash | 68.6% | 11.9% | −56.7 pts |
| UiPath ScreenAgent / GPT-5 mini | 77.0% | 18.4% | −58.6 pts |
| UiPath ScreenAgent / GPT-5 | 84.8% | 19.4% | −65.4 pts |
| Human baseline | 97.9% | 61.2% | −36.7 pts |
Source: arXiv:2511.17131, Tables 5 and 8, read 28 October 2026. Complex tasks include the paper's 50 copy-paste workflows and 40 enterprise application scenarios; per-category scores for copy-paste alone are not reported separately in the content we could access.
The gesture phoneshell already has for exactly this
iOS has its own built-in gesture for copy and paste that has nothing to do with menus: a three-finger pinch inward copies the current selection, and pinching outward pastes it, system-wide, in any text field. phoneshell/gestures.py implements it as three_finger_pinch(cx, cy, spread_out), a three-path W3C touch sequence converging on or diverging from a point, and wraps it in two one-line convenience methods, copy_selection(cx, cy) and paste_selection(cx, cy). The code has existed and worked since before this run; it is not a stub. What does not exist anywhere in environments/ is a task that calls it. Two of the 80 files touch the neighborhood: notes_text_select.yaml asks an agent to long-press a word until "Select All, Look Up, Paste, Cut" appears on screen, and safari_link_menu.yaml asks for a link's preview menu to show "Copy Link" among its options. Both check only that the menu's text appeared; neither task performs the pinch, and neither moves anything from one place to another, let alone one app to another.
What it means for a harness with the move but no reason to use it
- Having the gesture is not the same as having tested it.
three_finger_pinch's own geometry, three simultaneous paths converging over roughly 300 milliseconds, has never been checked against a real cross-app paste on a physical iPhone inside a scored task, so whether it reliably lands at UI-CUBE's task scale is unverified in either direction. - UI-CUBE's attributed cause, memory management, hierarchical planning and state coordination across apps, is a model-side claim about why complex tasks fail. phoneshell's gap is upstream of that: there is no task yet that would even give a model the chance to fail at coordinating a copy from one app into a paste in another, because no task asks it to.
- The two existing tasks that get closest,
notes_text_select.yamlandsafari_link_menu.yaml, are both taggeddifficulty: hardfor reaching a context menu via a sustained long-press, not for anything about moving data. phoneshell's current notion of "hard" in this area is about gesture timing, not about the cross-app state UI-CUBE's collapse is attributed to.
Check it yourself
# the working gesture, unused by any task
grep -n "def three_finger_pinch\|def copy_selection\|def paste_selection" -A4 phoneshell/gestures.py
# confirm no environment task calls it
grep -rl "copy_selection\|paste_selection\|three_finger_pinch" environments/*.yaml || echo "no task calls it"
# the two tasks that only check for menu text
cat environments/notes_text_select.yaml environments/safari_link_menu.yaml
# UI-CUBE's own task counts and Table 5 / Table 8
curl -s https://arxiv.org/html/2511.17131 | grep -iE "copy-paste|Table 5|Table 8"
What would prove this wrong
We claim phoneshell's three-finger pinch has never been exercised by any task in its public suite, and that its two copy/paste-adjacent tasks test only that a menu appeared, not that anything moved. That is wrong if a private or since-added task calls copy_selection or paste_selection and we did not find it in this grep of environments/. We also claim this gap sits upstream of UI-CUBE's attributed cause for its own collapse, meaning phoneshell cannot currently say whether its agents would fail at cross-app state coordination the way UI-CUBE's did, only that nothing has asked them to try. That is settled the day a phoneshell task is built that copies a value from one real app and pastes it into a second one, graded from device state, and either passes reliably or reveals the same kind of collapse UI-CUBE reports. We will treat that specific run, by either side, as the relevant check, by 30 April 2027.
Sources
- Cristescu, Park, Nguyen, Talmacel, Ilie, Adam, UI-CUBE: Enterprise-Grade Computer Use Agent Benchmarking Beyond Task Accuracy to Operational Reliability, arXiv:2511.17131, submitted 21 November 2025, read 28 October 2026.
- Full text, Tables 5 and 8: arxiv.org/html/2511.17131v1.
- UiPath, UI-CUBE benchmark, UiPath AI Research.
- Blolabel, phoneshell:
phoneshell/gestures.py,environments/notes_text_select.yaml,environments/safari_link_menu.yaml, read 28 October 2026.