A11y-CUA: Claude Sonnet 4.5 drops from 78.3% to 28.3% success when a desktop screen is magnified; phoneshell's own vision gap is 100% or 0%, never in between

A11y-CUA runs two computer-use agents through 60 Windows tasks under three levels of restricted access and gets a slope. phoneshell ran a pre-registered probe on iOS varying one channel at a time and got a step function. Both are real; they are measuring different things.

A11y-CUA, a CHI '26 dataset paper (DOI 10.1145/3772318.3791896, arXiv:2602.09310, submitted 10 February 2026), scores Claude Sonnet 4.5 at 78.3% of 60 everyday Windows tasks with full mouse and screen access, 41.6% restricted to keyboard-only input, and 28.3% viewed through a 150% magnifier. Qwen3-VL-32B-Instruct scores 20.0% with full access and 0.0% under either restriction. phoneshell's own experiment 2, run on a physical iPhone and pre-registered before the data existed, varies exactly one channel, the screenshot, and finds no slope at all: 100% with it, 0% without it, on pages built so the answer is rendered into an image, and 97% to 100% either way on pages where the answer is already text in the accessibility tree. The two results do not contradict each other. They show that "how much does vision loss cost an agent" has two different correct answers depending on whether you are averaging over many tasks with mixed channel dependence, or asking the question one screen at a time.

What changed, and when

A11y-CUA was built by Ananya Gubbi Mohanbabu, Rosiana Natalie, Brandon Kim, Anhong Guo and Amy Pavel, and accepted to CHI '26 (Barcelona, 13 to 17 April 2026). It recorded 16 participants, 8 sighted users and 8 blind or low-vision users, completing 60 everyday Windows tasks: 40.4 hours and 158,325 logged events, captured with an open-source recorder the authors built that keeps screen video, input events and the accessibility tree together. Against that human baseline they ran two current computer-use agents under three conditions: Default-CUA (full mouse and keyboard, full-resolution screen), SR-CUA (keyboard-only navigation, the condition a screen-reader user is closest to), and Magnifier-CUA (a 150% zoomed viewport, the condition a low-vision magnifier user is closest to).

phoneshell's experiment 2 is a different design answering an adjacent question, run earlier in 2026 and documented in docs/EXPERIMENT-2.md before any run happened. Twenty tasks, three vision-capable models (GPT-5.1, Gemini 3.1 Pro, Kimi K3), each task run twice, once with the screenshot and once without, order alternated, 120 runs in 37 minutes on a physical iPhone. Ten of the twenty tasks are a controlled stimulus: one word written as real text in the page and a different word rendered into an image with no alt text, verified on device so that 0 of 10 image words appear in the accessibility tree and 10 of 10 written words do. The other ten tasks ask about content that is already labelled, as a control.

The two tables, side by side

Read down the A11y-CUA column and the number falls in steps as access narrows. Read down the phoneshell column and there is no fall at all, only a jump that depends on which stratum you are in. McNemar's exact test on phoneshell's render stratum gives p < 0.0001 on a 30-0 split; the tree stratum, held as a control, gives p = 1.0 on a 0-1 split. Both numbers are in the same paragraph of the same source because the point is the contrast between them, not either one alone.

What this means for someone building or evaluating a phone or desktop agent

An average success rate under a restricted-access condition answers "across a realistic mix of everyday tasks, how much does this restriction cost." That is the right question for accessibility conformance and for predicting a deployed agent's real-world degradation. It is the wrong question for deciding whether to pay for vision on a specific screen, because the average is itself a mixture of tasks at 100% and tasks at 0%, and the mixture ratio, not the mechanism, is what moves the headline number. phoneshell's render-stratum result says the floor under those 0% tasks is a hard floor, not a soft one: there is no amount of model capability in these three vision-capable frontier models that recovers a word that exists only as pixels with no text channel beside it. A11y-CUA's Magnifier-CUA condition at 28.3% is consistent with a dataset that contains a mix of tree-exposed and pixel-only targets in roughly that ratio, though the paper does not break its 60 tasks down that way and we have not re-sliced it ourselves.

The practical rule carried over from phoneshell's own finding (FINDINGS.md §26) still applies across platforms: decide per screen, not per deployment. A screen whose content is already exposed to assistive technology costs nothing extra to automate without vision, on either Windows or iOS. A screen that bakes its answer into pixels, a map tile, a chart, a scan, costs everything. An aggregate percentage under one access condition cannot tell you which kind of screen you are looking at; only a per-screen audit, of the kind render_gap.py runs on this project, can.

Check it yourself

A11y-CUA's own numbers are in its CHI '26 proceedings PDF and the arXiv preprint; neither the authors' dataset recorder nor a benchmark harness has a public release we could find as of this writing, so the task-by-task breakdown behind the aggregate percentages is not independently reproducible from outside the paper. phoneshell's half of the comparison is public end to end:

git clone https://github.com/Blomega/phoneshell && cd phoneshell
cat docs/EXPERIMENT-2.md          # pre-registered design, written before the run
grep -A4 "render stratum" FINDINGS.md   # section 26, the 30/30 vs 0/30 split
python3 scripts/render_gap.py --help    # the per-screen OCR-vs-tree audit this argument depends on

What would prove this wrong

• If A11y-CUA's released task list turns out to contain no pixel-only targets at all, the 28.3% and 0.0% magnifier scores would have to come from a different mechanism than the one we propose here (channel overlap, not a binary floor), and the cross-platform analogy we are drawing would not hold. We have not seen the task-by-task list and would want to before treating this prediction as settled.
• We predict that re-slicing A11y-CUA's 60 tasks by whether the target text is present in the Windows accessibility tree under Magnifier-CUA, the same way phoneshell's render and tree strata were split, would produce two groups separated by close to the full 100-point range, not a uniform 28.3% across both. This is falsifiable by anyone with access to the dataset and we have not run it ourselves, through 31 December 2027, the authors' own data-sharing window implied by the CHI '26 publication date.
• The blind/low-vision human baseline is reported as both 84.6% and 86.9% in different parts of the same paper. We flag this as unresolved rather than picking one; if the discrepancy turns out to be a typo correctable from the authors' own supplementary material, the human-vs-agent gap narrows by two points and nothing in this article's comparison changes.