Masking screenshots cut PriMobiBench's profiling by a third, not 58%, and a tree-reading agent sends the text anyway

The paper's own Table 5 and Table 6, reproduced row by row, and a read of what one tree-first iPhone harness sends to a cloud model on every step.

PriMobiBench (arXiv:2609.13873, 12 September 2026) reports that masking private regions of a phone screenshot cuts a model's ability to profile the user "by up to 58% with only ~8% performance loss". The two numbers come from different rows. The 57.9% cut is Qwen2.5VL-7B under a mask that costs agents 15.1% to 20.4% of their action consistency; the mask that costs about 8% cuts GPT-5-mini's profiling by 32.1%. And every model in the study saw "only raw screenshots without metadata", so the channel a tree-reading iPhone agent actually leans on, text from the accessibility tree, was never masked or measured.

What PriMobiBench measured

Qihang Cen, Tianshuo Cong, Da Song, Xinlei He and colleagues (Tsinghua University, Shandong University, Wuhan University) built MobiLeak: 50 synthetic user profiles, 25 privacy attributes in 5 categories, inserted into trajectories from 16 Android apps collected by Mobile-Agent-v3. That gives 320 samples carrying 2,960 privacy elements in 30 visual forms. Five vision-language models were asked to pull private facts off the screenshots (recognition) and to assemble a profile of the user across a trajectory (profiling). A separate set of manually collected iOS 26 trajectories, count not reported, gave GPT-5-mini 0.786 recognition and 0.652 profiling.

The mitigation runs on the client before anything reaches a cloud model. Qwen3-VL-8B segments the screenshot into UI elements, classifies which ones hold private data, and pixelates them. The "task-relevance filtering" variant leaves private elements alone when the task needs them. The paper's Table 5 and Table 6 carry the results; we reproduce the rows the abstract draws on.

Read across a row and the trade is clear. The configuration a deployer would actually choose, segmentation plus task filtering, costs agents 6.3% to 10.0% and removes 30.8% to 46.6% of profiling success. The strongest model keeps about two thirds of its profiling ability: 0.472 against 0.695. That is a real reduction and worth having. It is not 58%. A second wrinkle: GPT-5-mini's unmasked recognition is 0.825 in the mitigation table and 0.747 in the paper's main recognition result, so the abstract's "up to 82.5%" is the higher of two baselines for the same model.

Where a pixel mask sits in a tree-first iPhone agent

PriMobiBench's threat model is a screenshot agent, and its related-work section notes that some agents also read accessibility trees without testing one. On iOS that is the common case, because most first-party and many third-party apps expose a rich tree. Our own harness is built that way, and reading its observation code shows what a screenshot mask would and would not cover.

Concretely: to_tsv writes one row per element with its type, up to 90 characters of its text, its centre, its size and its flags, and that text goes out on every step. The picture is downscaled to 512 pixels on its long edge and encoded as a quality-70 JPEG, and the MCP server sends it once per distinct screen per session. When the tree is too thin to trust, blind mode runs macOS Vision on the Mac, which is local, and appends what it reads to the same TSV. So on the screens where pixels matter most, the words in the pixels have already become text before any image-side mask could act. The only protection in the step path belongs to iOS: a secure text field does not expose its contents. safety.py adds a deny list of three bundles (Passwords, the Settings passwords pane, Keychain) that the agent may not open, which prevents some screens rather than redacting any.

What this means if you ship a phone agent

Masking screenshots is the right control for a screenshot-only agent, and PriMobiBench's element segmentation beats a coarse mask on both axes: less leakage and less task damage in every row of Table 6. For an agent that also reads the tree, it is half a control. The redaction has to happen on the element list, before it is serialised, and the same classifier decision has to drive both the TSV row and the pixel region, or the two channels disagree about what is private.

That makes the tree the easier place to redact, not the harder one. A row already carries its type, its text and its rectangle, so there is no segmentation step to get wrong. Masking a row's text and the pixels inside its rectangle together is one decision per element. What it does not solve is inference: PriMobiBench's profiling scores are about assembling a person from many harmless-looking facts over a trajectory, and no per-element rule sees that.

Our own position, stated as a judgement: phoneshell today sends everything the tree exposes, and runs on a personal phone. We have not measured how many private facts a typical task's TSV carries. That is the number to get before arguing about masks.

Check it yourself

# the paper's tables (Table 5: masking vs recognition and profiling; Table 6: agent cost)
curl -sL https://arxiv.org/html/2609.13873v1 -o pmb.html
python3 -c "import re,html;t=html.unescape(re.sub(r'<[^>]+>',' ',open('pmb.html').read()));
i=t.find('Table 5. Privacy');print(re.sub(r'\s+',' ',t[i:i+1600]))"
grep -o 'only raw screenshots without metadata' pmb.html

# what phoneshell sends per step
git clone https://github.com/Blomega/phoneshell && cd phoneshell
grep -n "def to_tsv" -A10 phoneshell/agent/observation.py     # 90-char text per element
grep -n "max_edge\|quality=70" phoneshell/agent/observation.py
grep -n "seen_screens" phoneshell/mcp_server.py                 # one picture per screen
grep -n "denied_bundle_ids" -A6 phoneshell/config.py            # three bundles
grep -rn "redact\|mask" phoneshell/agent phoneshell/perception/tree.py   # nothing

What would prove this wrong

The claim is that screenshot masking leaves a tree-reading agent's leakage mostly intact, because the same private text travels in the accessibility tree. It is wrong if, on iOS, the tree carries little of what the pixels show: if most private facts in real apps live in images, web views or custom-drawn text that the tree does not expose. We will run PriMobiBench's 25-attribute taxonomy against phoneshell's own TSV and JPEG logs for the 80 public tasks and publish, by 15 December 2026, the share of private facts that appear in the TSV, in the JPEG only, and in both. If the TSV share is below a third, pixel masking covers most of the exposure on iOS and this note is wrong.