Instant screenshots made a computer-use agent 10.6% slower: it looked before the screen had redrawn

cua-speedrun made the environment answer in milliseconds and the agent got slower: 89.54 to 99.02 seconds per task, with 40% more action batches, at the same score. A fast screenshot of a screen that has not redrawn is a wasted turn. phoneshell waits about 0.9 seconds before it looks, which avoids most of this and still loses the race to any app slower than that.

Making a computer-use environment answer in milliseconds instead of seconds made the agent slower. In cua-speedrun, released on 30 September 2026, swapping the standard runtime for FastCUA cut environment time on 50 OSWorld tasks from 13.80 to 5.08 seconds, and GPT-6 Astra (low, via Codex) still took 99.02 seconds per task instead of 89.54, at a score of 90.42% against 90.82% (arXiv:2609.40284, Tables 4 and 11, five seeds). The screenshots came back before the interface had redrawn, so the agent spent turns asking again.

cua-speedrun times agents on one setup instead of many

The paper is by Pranjal Aggarwal, Lawrence Keunho Jang, Sean Welleck, Daniel Fried, Ruslan Salakhutdinov and Jing Yu Koh. Its premise is that speed numbers from existing benchmarks cannot be compared, because every lab runs different machines and containers. cua-speedrun runs every agent through one interface (observe, step(actions), done) against QEMU virtual machines on Modal at 1920x1080, starts the clock at the instruction and stops it at termination, and splits each task's time into agent time and environment time. Setup and verification are excluded. Infrastructure failures are retried; agent failures are scored.

It covers four benchmarks, each cut to a subset: OSWorld (295 tasks to 50), OSWorld 2.0 (63 to 52), CUA-World (143 to 26) and MyPCBench (184 to 38). The subsets are chosen to keep the ranking of agents: on held-out agents, the 50-task OSWorld subset ranks them with a Spearman correlation of 0.983 on partial score and an error of 2.16 points, against 4.02 points for a random 50. The paper calls this preserving statistical power; it shows rank agreement and error, not a power calculation. The project site is cuaspeedrun.com; the GitHub repository it links was created on 1 October 2026, a day after submission, and carries no licence file.

The environment saved 8.72 seconds per task; the agent spent 18.20 more. Batches rose 40% and cost 71%. The agents also asked for four times as much explicit waiting (4.74 against 1.12 seconds), which is the model rediscovering the sleep the runtime used to provide. The paper's example: after opening Save As, the first screenshot shows no dialog, and the agent asks for another one 5.8 seconds later. The authors estimate a wait of about 100 milliseconds would have done, and say current models cannot judge it.

More reasoning was faster for one model and slower for another

The second result that runs against intuition: Gemini 3 Flash Preview on the 50 OSWorld tasks scored 33.6% in 492.2 seconds at low effort and 57.6% in 268.0 seconds at medium, and medium was also cheaper ($0.592 against $1.401 per task, Table 8). Low effort produced long runs of repeated failed actions, such as trying to enter spreadsheet text with key combinations. The text quotes this as 56.0 against 32.5 steps; those figures match the table's model responses column, while the column the appendix calls steps (action batches) reads 39.52 against 28.26.

The effect does not generalise. Gemini 3.8 Flash scored 91.6% at both low and medium effort, and medium took 250.1 seconds against 127.1. On the harder OSWorld 2.0, going from medium to high bought 0.3 points (59.9 to 60.2%) for 49% more time. So the claim to carry away is narrow: reasoning effort and wall time are not monotone, and the direction depends on whether the cheap setting loops.

A phone agent has the same race on a longer fuse

phoneshell never looks immediately. After an action, _after in phoneshell/actions.py sleeps 0.6 seconds and calls the action effective if more than 2% of pixels changed. The next observation then goes through wait_until_stable in phoneshell/perception/screen.py: screenshots every 0.3 seconds until two consecutive frames differ by at most 1.2%, capped at 3.5 seconds, then one accessibility tree read. The snapshot docstring gives the reason in numbers: a home-screen read 1.0 second after pressing home returned 6 elements, the same read at 2.0 seconds returned 12.

That design avoids the FastCUA failure for anything that starts moving inside about 0.9 seconds. It does not avoid it for an app that waits on the network before redrawing. If the first two probes match because nothing has started yet, the gate closes on the old screen, the signature is unchanged, and _act_and_report in phoneshell/mcp_server.py tells the model "THE SCREEN DID NOT CHANGE ... try something different rather than repeating it". That is the paper's mechanism with a 0.9 second fuse instead of a 20 millisecond one, and it pushes the model to change course on an action that worked. This is our reading of the code; we have not measured how often it fires.

What it means for anyone timing phone agents

The per-step budget in the phoneshell/memory.py docstring, measured on a real iPhone, is a tree read of 159 ms, a screenshot of 72 ms, a stability gate of 504 ms, a fixed settle of 600 to 800 ms, and 1 to 3 seconds for the model. The device side is therefore 1.3 to 1.5 seconds a step, comparable to the model. cua-speedrun's result says shaving that side is only a win if the agent is told when the screen is ready. A cheaper wait that hands the model a stale screen moves the cost into turns, which are the expensive part.

The paper's other claim, that no model family is best on score, time and cost at once, holds on our phone too. On the 9 September 2026 sweep (58 paired tasks, iPhone 14 Pro, iOS 26.3.1), GPT-5.1 was fastest at 37.7 seconds a task, Kimi K3 passed the most at 96.6%, and DeepSeek V3.2 was cheapest at $0.1451 for all 58 tasks and slowest at 68.3 seconds. Per turn, the six models range from 7.8 seconds (Qwen3 Max, 43.4 s over 5.6 turns) to 13.0 seconds (Kimi K3, 66.5 s over 5.1 turns), our division of the board averages. One run per model and task, so treat the order as an anecdote; none of the pairwise pass-rate differences on this board is significant, as an earlier note showed.

Decisions that follow. Record agent time and device time separately per step; phoneshell's results file stores only seconds and turns per task, so today we cannot say how much of the 37.7 to 68.3 seconds is the phone. Treat the post-action wait as part of the action contract, not a tuning constant: an action should return when the screen is ready, or say that it timed out. And when comparing models on speed, compare at the same wait policy, because the wait changes the number of turns as much as the model does.

Check it yourself

• The paper: https://arxiv.org/abs/2609.40284. Table 2 (standard per-action waits), Tables 1 and 3 (FastCUA latency), Table 4 (FastCUA runs), Table 11 (standard runs, five seeds), Tables 7 and 8 (OSWorld results with time and cost), Section 3.2 (Save As example), Table 14 (subset validation).
• The code: https://github.com/cua-speedrun/cua-speedrun; licence status: curl -s https://api.github.com/repos/cua-speedrun/cua-speedrun | grep -A2 license returned null on 3 October 2026.
• phoneshell's waits: grep -n "def _after" -A8 phoneshell/actions.py (0.6 s, 2%), grep -n "def wait_until_stable" -A6 phoneshell/perception/screen.py (0.3 s, 1.2%, 3.5 s), grep -n "THE SCREEN DID NOT CHANGE" phoneshell/mcp_server.py.
• The leaderboard figures: python3 -c "import json;b=json.load(open('site/results.json'))['board'];[print(m,v['avg_seconds'],v['avg_turns'],v['pass_rate'],v['total_cost_usd']) for m,v in b.items()]" from the repository root.

What would prove this wrong

By 31 March 2027 we will add per-step agent and device timing to phoneshell's results and log every "THE SCREEN DID NOT CHANGE" reply with the pixel difference measured 3 seconds later. We predict two things. First, device time will be at least 25% of task time for every one of the six models. Second, at least 1 in 20 unchanged replies will turn out to be late responses, where the screen did change within 3 seconds. If device time is under 25% for most models, the settle is not worth optimising; if fewer than 1 in 20 unchanged replies are late, the 0.9 second fuse is long enough for the apps we test.