MyPCBench's zero-score trajectories still average 22 to 85 steps per task; phoneshell's fix for a lock screen that ends a run, built after losing four of them, is wired into one feature and not the benchmark

MyPCBench's six agents average 43.7 to 69.2 steps per task, and even the runs that score zero do not quit quickly: failed trajectories from three models still average 22 to 31 steps before giving up, and two others average 52 to 85. phoneshell already has code for the one thing a long, idle-heavy run risks on a real iPhone, the screen locking mid-task, built because "this project has lost four runs that way." That fix is called from exactly one feature. It is never called from the code that runs phoneshell's own 80 scored tasks.

MyPCBench (arXiv:2606.16748, Lawrence Keunho Jang, Andrew Keunwoo Jang, Jing Yu Koh and Ruslan Salakhutdinov, submitted 15 June 2026) builds 184 tasks around one persona, Michael Scott, seeded consistently across 17 simulated web applications on a Linux desktop. Its Table 2 reports both a "Perfect" rate, the share of tasks every rubric item passed, and average steps per task; its best model, Claude Opus 4.6, needs 46.5 steps on average and solves 55.4% of tasks perfectly. Every model tested needs dozens of steps per task, and the paper's own zero-score breakdown shows that failing is not a fast process either. phoneshell runs its own 80 tasks with a 240-second timeout and no step count anywhere near MyPCBench's, but it shares the one physical property every long automated session on a real phone has to survive: the screen has to stay on.

How long MyPCBench's tasks actually run, including the ones that fail

MyPCBench grades two ways: "Perfect," a full rubric pass, and a weighted "Rubric score" giving partial credit, alongside average steps and a trajectory-efficiency figure, rubric points per step. The six models tested span a wide range on all four. What stands out for a question about session length is the step counts on the zero-score end: the paper's own text states that zero-score trajectories from GPT-5.5, GPT-5.4 mini and Claude Sonnet 4.6 average about 22 to 31 steps, meaning those models quit early, while Opus 4.6 and the two Qwen 3.5 models average 52 to 85 steps on their own zero-score runs, quitting late or not at all before the task's own cap. Figure 6 in the paper plots a 100-step budget curve on which Opus is still rising at the cap.

Source: arXiv:2606.16748, Table 2, read 28 October 2026. 184 tasks, Linux desktop, 17 simulated web apps seeded with one persona's accounts and history.

The fix phoneshell built, and the one place it is called from

phoneshell/autolock.py states its reason for existing in its own opening lines: "A lock ends a scan. iOS refuses to launch apps from the lock screen, so the run stops producing screenshots and reports a failure that is really just a phone doing its job. This project has lost four runs that way." Its fix, a context manager called KeepAwake, reads the phone's current Auto-Lock setting off the Display & Brightness row text (no need to open the Auto-Lock screen itself), sets it to Never for the length of a scan if it is not already there, and restores the original value on every exit, including a crash, because leaving someone's personal phone set to never sleep "would be a rude thing to do with their battery." The class verifies the change took by reading the setting back rather than trusting the tap, the same principle autolock.py's own docstring states for reading it: "a tap that reports success and changes nothing is the normal failure in this stack, not the unusual one."

A repository-wide search for KeepAwake finds exactly two files: its own definition in autolock.py, and one call site, in phoneshell/crawl.py, where it wraps the app-crawling feature's own scan. phoneshell/bench/runner.py, the module that actually executes phoneshell's 80 scored tasks against a model, one at a time, up to 240 seconds each, contains no reference to KeepAwake or autolock at all, and neither does cli.py, server.py or mcp_server.py. No task YAML, no setup step and no documentation we found tells an operator to set Auto-Lock to Never by hand before running the benchmark either. The fix built specifically because of four lost runs protects a different feature than the one phoneshell measures itself by.

What it means for a benchmark that has already lost runs to this

• MyPCBench's own numbers describe a different device and a different failure surface, a Linux desktop with no screen lock to speak of. The relevance to phoneshell is not that MyPCBench's exact step counts transfer to an iPhone, it is that MyPCBench's agents, and most agentic harnesses at this task length, spend dozens of steps per task including the ones that end in failure, and every one of those steps is an opportunity for an idle gap on a real phone to run past whatever Auto-Lock interval is currently set.
• phoneshell's own default task timeout, 240 seconds, is itself evidence the risk is not zero: a model response that takes more than a typical iOS Auto-Lock interval (30 seconds up to 5 minutes, device-dependent) between taps, which the project's own cost comments elsewhere describe as "1-3s... everything else is noise" but do not bound on the slow end, would pass that interval with nothing touching the screen.
• The fix already exists, tested, verified, and safe to leave running unattended because it restores the original setting on every exit including a crash. Wiring the same context manager around bench/runner.py's per-task loop is not a new feature; it is calling a function that already works from a second place.

Check it yourself

# the reason KeepAwake exists, in its own words
sed -n '1,14p' phoneshell/autolock.py

# every file that references it
grep -rn "KeepAwake" --include="*.py" .

# confirm bench/runner.py has no reference at all
grep -c "KeepAwake\|autolock" phoneshell/bench/runner.py

# phoneshell's fixed per-task timeout, applied to all 80 tasks
grep -n "timeout_seconds" phoneshell/bench/schema.py
grep -h "timeout_seconds" environments/*.yaml | sort -u || echo "no task overrides the default"

# MyPCBench's own step and zero-score figures
curl -s https://arxiv.org/html/2606.16748 | grep -iE "average steps|zero-score|Table 2"

What would prove this wrong

We claim KeepAwake is called from exactly one place in phoneshell's own source, crawl.py, and never from bench/runner.py, so phoneshell's own 80 scored benchmark tasks run with no automatic protection against the same lock-screen failure mode the class exists to prevent. That is wrong if a later commit wires it into the runner and we checked a stale copy of the source. We also claim this is a real, not theoretical, exposure given phoneshell's own 240-second timeout and the range of iOS Auto-Lock intervals. That claim is falsified the day someone runs phoneshell's benchmark suite with Auto-Lock deliberately left at a short interval, say 30 seconds, across enough tasks to see whether any fail specifically because the screen locked mid-task, a test nothing in the current results schema would currently let anyone attribute correctly even if it happened. We will treat either event, a code change or a reproduced lock-related failure, as grounds to revisit this piece by 30 April 2027.