MobiFlow's own text and its own table disagree about which model scores 60.4; phoneshell never needs the trajectory graph MobiFlow builds to referee that argument

MobiFlow's introduction and its Table 3 agree that UI-TARS-1.5 scores 60.4% and GUI-Owl scores 55.7%. Section 5.2 of the same paper instead credits that 60.4 figure to GUI-Owl, under a different metric than the one Table 3 defines. The two readings cannot both be right, and the paper does not say which one is. phoneshell never has to adjudicate a result like this, because it never needs the trajectory-fusion graph MobiFlow built to get a result in the first place.

MobiFlow, posted to arXiv as 2604.09587 by a team including researchers at Shanghai Jiao Tong University, starts from a real constraint: AndroidWorld-style benchmarks score a task by reading system-level emulator state, and most third-party apps simply do not expose that state the way the handful of open-source reference apps those benchmarks lean on do. MobiFlow's answer is to record multiple human trajectories through the same task on a real third-party app (the paper's own example: 7 recorded paths cover every valid way to complete a roughly 15-step Weibo-comment task) and fuse them into a single state-transition graph, which becomes the thing a model's run is checked against. Across 20 third-party apps and 240 real-world tasks, the paper reports UI-TARS-1.5 at 60.4% and GUI-Owl at 55.7% success rate in its base scenario, in both the introduction and Table 3.

What changed, and when

Section 5.2 of the same paper states that "GUI-Owl attains the highest completion rate (CR) of 60.4." That sentence assigns the number 60.4 to GUI-Owl and to a metric called completion rate. But Table 3 lists 60.4 as UI-TARS-1.5's success rate (SR), and gives GUI-Owl's own completion-rate column a different value entirely, 77.9. Read literally, the paper's prose and its own table disagree about both which model earns the 60.4 figure and which metric it belongs to. We read this from the sections of the HTML version available to this check on 24 October 2026 and could not find a correction or an explanation for the mismatch elsewhere in the text; it may be a labeling slip in the prose, a table that was updated after the prose was written, or something else, and we are stating that plainly rather than picking the reading that makes a cleaner headline.

Fusing trajectories to see state phoneshell just reads off the phone

MobiFlow's graph-fusion method exists for a specific reason: an emulator running a third-party app usually cannot tell a benchmark whether a task actually succeeded, because the app's own internal database is opaque from outside. Recording several real completions of the same task and fusing them into a state-transition graph gives the benchmark something to check a new run against, without needing the app's cooperation. phoneshell's checks (phoneshell/bench/schema.py's Check class) do not face this problem the same way, because phoneshell is not driving an emulator against an app that might hide its state. It drives one physical iPhone and reads the operating system's own signals directly: which app is in the foreground, what a Settings toggle's value is, what text is on screen, each as a named check kind rather than an inferred position in a fused graph. Where MobiFlow has to reconstruct what "done" looks like from multiple recordings because the app itself won't say, phoneshell can usually just ask the phone.

What it means for anyone reading a benchmark's own numbers

• A number that appears twice in the same paper is not automatically the same claim twice. 60.4 shows up once as UI-TARS-1.5's success rate in Table 3 and once as a description of GUI-Owl's completion rate in Section 5.2's prose; treating those as corroborating each other, rather than checking which table column and which model each actually belongs to, would understate how uncertain the paper's own headline comparison is.
• Trajectory fusion is a reasonable answer to a real constraint (most third-party apps hide their state from an emulator), and it is also a method with its own failure surface: a graph built from 7 human paths through one task is only as complete as those 7 paths, a judgment call the paper does not quantify with an error bound.
• Running on a real device instead of an emulator does not make phoneshell's checks infallible; it changes what has to be trusted. phoneshell trusts the operating system's own accessibility tree and settings APIs instead of trusting that a handful of recorded trajectories cover every way a task could legitimately be completed.
• Grading from device state, phoneshell's own house rule, is not a generic virtue independent of what state is actually available to read. It works here because iOS exposes a real accessibility tree and settings values to a connected tool; MobiFlow's method is what a benchmark has to build when that kind of direct read is not available at all.

Check it yourself

# MobiFlow's own Table 3 vs. its Section 5.2 description
curl -s https://arxiv.org/html/2604.09587 | grep -i "60.4\|UI-TARS-1.5\|GUI-Owl\|completion rate"

git clone https://github.com/Blomega/phoneshell && cd phoneshell
grep -n "class Check\|kind:" phoneshell/bench/schema.py          # named check kinds, no fused graph
grep -rn "foreground_app\|settings" phoneshell/bench/ | head -5  # checks read OS state directly

What would prove this wrong

We claim MobiFlow's Table 3 and its Section 5.2 text assign the number 60.4 to two different models under two different metrics, with no reconciliation in the sections we read. That is wrong if a later revision of arXiv:2604.09587 corrects or explains the mismatch, which we will check by 30 April 2027. We also claim phoneshell's checks read real device OS state directly rather than inferring state from a fused trajectory graph; that is wrong the moment any check in phoneshell/bench/schema.py or its verifier does the latter, and we found none as of 24 October 2026.