EntWorld's Table 3 has Qwen2.5-VL-32B losing 4.9 points when a screenshot joins the tree; phoneshell's own docstring cites only the row that gained 18.5

EntWorld (arXiv:2601.17722, 25 January 2026) ran the same screenshot-plus-tree ablation on three models across six enterprise apps. Combining inputs added 18.5 points for a model fine-tuned on the benchmark's own apps and cost 4.9 points for the untuned model in the same table. phoneshell's observation code quotes only the first number, uncredited, to justify a design it ships to models that look like the second.

phoneshell's own observation code justifies its design with one parenthetical: "Hybrid beats either channel alone (53.7% vs 35.2% tree-only vs 15.6% pixels-only)," the first of three measured results the module's docstring says shaped the choice to feed a model both the accessibility tree and a numbered screenshot overlay on every step. Those three numbers come from EntWorld (arXiv:2601.17722, submitted 25 January 2026 by Ying Mo, Yu Bai, Dapeng Sun, Yuqian Shi, Yukai Miao, Li Chen and Dan Li, Zhongguancun Laboratory and Tsinghua University), and in the paper's own Table 3 they belong to one specific model: EntAgent-RL, fine-tuned by reinforcement learning on EntWorld's own six enterprise apps. The same table has a closer match to what phoneshell actually runs: Qwen2.5-VL-32B-Instruct, used off the shelf with no EntWorld-specific tuning at all, the way phoneshell runs every frontier model it grades. For that model, combining the screenshot with the tree did not help. It cost 4.9 points, from 27.2% tree-only down to 22.3% combined, in the same table phoneshell's code quotes from, without naming it.

What EntWorld measured, and on whom

EntWorld builds six enterprise applications, EspoCRM, ZenTao, OpenProject, Veops CMDB, iTOP and Snipe-IT, into 1,756 verifiable tasks checked against SQL-level state rather than visual matching, and its Table 3 runs an observation-modality ablation across three models and three input conditions: accessibility tree only, screenshot only, and both together. Two of the three models are EntWorld's own: EntAgent-SFT, supervised fine-tuned on the benchmark's own trajectories, and EntAgent-RL, further trained with reinforcement learning on the same apps. The third, Qwen2.5-VL-32B-Instruct, gets no EntWorld-specific training of any kind. The paper's text states plainly that the accessibility tree alone beats the screenshot alone, on average, for every model in the table, and that relying on the tree alone "is particularly inadequate for legacy enterprise applications, where accessibility metadata is often incomplete or inconsistently implemented." It does not say, in the prose, what happens to the untrained model when the two inputs are combined. Table 3 says it anyway.

Source: EntWorld, arXiv:2601.17722, Table 3, averaged across EspoCRM, ZenTao, OpenProject, Veops CMDB, iTOP and Snipe-IT. Per-app figures in the paper range from 4.7% to 70.0%.

Which row phoneshell actually looks like

phoneshell/agent/observation.py builds exactly the input EntWorld's "combined" column tests: a numbered set-of-mark screenshot plus a TSV-serialized accessibility tree, sent to the model on every step. The module never fine-tunes the models it evaluates. Every model phoneshell grades, through the OpenRouter path or the Claude Code CLI path, is run the way EntWorld ran Qwen2.5-VL-32B-Instruct: off the shelf, with no EntWorld-style training on phoneshell's own 80 tasks or any other. Nothing in the repository runs reinforcement learning on phoneshell's task set, and nothing could, since the published suite deliberately withholds a private task set from bench/schema.py's own loader specifically so a model generation cannot memorize the answer key. The model type EntWorld shows losing 4.9 points from the same combination phoneshell ships by default is, architecturally, the only model type phoneshell has ever run.

What it means for a design choice that quotes a benchmark

• The docstring's number is accurate. 53.7% beats 35.2%, which beats 15.6%, exactly as written. What the comment omits is which row that is, and that the same table's untrained row moves the other way.
• Enterprise web forms and native iOS apps are not the same rendering surface either. EspoCRM, ZenTao and the other five apps run inside a browser-rendered desktop UI; phoneshell's tasks run against native iOS apps through XCUITest. A result's direction can fail to transfer across that gap even before the tuning mismatch is considered.
• This is not evidence that phoneshell's hybrid design is wrong. EntWorld's own text says the tree beats the screenshot alone for every model, tuned or not, which supports leading with the tree the way phoneshell's docstring already does. The specific number cited for the combination, though, needs the kind of fine-tuning phoneshell does not do and currently has no mechanism to do.
• No citation, to EntWorld or anything else, appears anywhere in phoneshell's repository next to this number. A reader of the docstring alone has no way to find Table 3, or the row next to the one quoted.

Check it yourself

# the uncredited number, and the design it justifies
grep -n "Hybrid beats either channel" -A2 phoneshell/agent/observation.py
sed -n '1,14p' phoneshell/agent/observation.py

# confirm no citation to EntWorld or arXiv:2601.17722 exists anywhere in the repo
grep -rniE "entworld|2601.17722" phoneshell/ README.md docs/ 2>/dev/null || echo "no matches"

# confirm phoneshell never fine-tunes or RL-trains the models it grades
grep -rniE "fine.?tun|reinforcement learning|\bRL\b.*train" phoneshell/bench/ phoneshell/agent/ 2>/dev/null || echo "no matches"

# EntWorld's own Table 3 and the claim about legacy accessibility metadata
curl -s https://arxiv.org/html/2601.17722v1 | grep -i "legacy enterprise\|Table 3"

What would prove this wrong

We claim phoneshell's observation design quotes EntAgent-RL's row from EntWorld's Table 3 without naming the paper, and that the architecturally closer row in the same table, for the untuned model, moves in the opposite direction. That is wrong if a citation to arXiv:2601.17722 exists anywhere in the repository we did not find, or if EntWorld's Table 3 is revised in a later arXiv version to show Qwen2.5-VL-32B-Instruct gaining rather than losing from the combination. It is also an open question, not yet tested by anyone, whether the direction phoneshell actually depends on, screenshot-plus-tree versus tree-alone, holds for the specific frontier models phoneshell grades on its own 80 iOS tasks. We will treat a phoneshell run that isolates tree-only against its current default as a direct test of that question, and revisit this piece if one is published before 30 April 2027.