Action-conditioned bisimulation stops GUI memories merging look-alike screens; phoneshell's layout fingerprint mostly splits them, 60 keys for 29 screens

The paper's warning is about merging too eagerly. We checked phoneshell's screen memory against its own crawl records and found the opposite error dominates, with one family of exceptions that a second paper this week tells explorers to skip.

Merging GUI states by what their actions do, instead of by how they look, raised a frozen Qwen3-8B from 72.3% to 78.2% success on ten MiniWoB++ tasks. Merging by appearance scored 70.8%. That is the result of a paper posted on 30 September 2026. phoneshell's screen memory merges by appearance. On its own eleven crawls of 11 September, the appearance key mostly erred the other way: 29 titled screens received 60 keys, 14 of them for Settings. Only 4 of 38 repeated taps behaved differently, and every one of the four was a promotional or banner row.

What the two papers found

Hongbo Zhang, Liuyang Song, Quanquan Li, Daqian Yang, Yan Wen and Zhengtao Yao (arXiv:2609.38778, "Action Conditioned Bisimulation For GUI Agent Memory", 30 September 2026) start from a specific failure: GUIs are full of pages that look alike and answer the same click differently, two tabs of one widget or two rows of one menu. Their rule merges two states only when shared actions lead to agreeing outcomes and successor blocks. It replaces only the merge rule of an existing outcome-value memory, so their ablation ladder isolates it. Each rung changes one thing and keeps the model, prompt, call budget and exploration detours fixed.

It is one seed, one model and web widgets, and the authors say so. The second paper is SAGE (Bo Han, Qianyi Wang, Shuai Liu and colleagues, arXiv:2609.39547, 30 September 2026). It argues that self-collected priors drift with app updates, promotions, ads, A/B tests and personalisation. Across 176 tasks in 20 apps with paired old and new versions, only 44% kept identical trajectories after an update. Its explorer assigns advertisements and promotional cards zero interaction budget.

phoneshell's fingerprint on its own crawl records

memory.fingerprint in phoneshell hashes the bundle id with the type, label and 32-point grid cell of every Button, Tab, Switch, text field, Link and similar control, plus the count of those controls divided by four. The crawler (crawl.py, line 918) stores the same hash as each screen's structural key and uses (structural key, control key) to decide that a control has been tried. The agent's memory uses it to decide when a screen is known. After two visits, observe() stops sending the screenshot, and the hint tells the model what each control "did last time".

The crawler's manifests record every tap with the before and after screen. Eleven crawls with manifests ran on 11 September 2026, eight of Settings and three of the Talika app: 113 screens and 237 edges, of which 193 taps reached another screen, 41 changed nothing and 3 left the app. Within one crawl a control is tapped once per layout, so repeated taps come from different crawls of the same screen. That makes it a test of the merge rule across time. We grouped edges by (source fingerprint, control) and compared what happened. Behaviour means the destination app and title, or no-op, or left the app.

The four disagreements are the paper's failure, but not in the paper's form. They are not two look-alike screens. They are one screen whose promotional row changed its effect between runs: the "You've reached your storage limit" banner left Settings in one crawl and stayed put in the next. "3 Months of Apple Music Included" opened "Services Included with Purchase" once and left the agent on a screen still titled Settings once. The Talika earnings card and header changed the screen in two taps of four and did nothing in the other two. An action-conditioned rule would refuse those merges. It would also learn nothing stable about them, which is SAGE's reason for not tapping them at all.

The larger error is splitting. A one-row scroll moves every row into another 32-point cell, and a list that grows by four controls changes the count term. So the screen a person calls Settings carried 14 keys, and 8 of the 38 repeated pairs landed on the same screen under a new destination key. The bisimulation rule would merge those, since their actions agree. For phoneshell, a split costs a memory that starts again from zero visits, so the screenshot keeps being sent and the hint stays empty. The paper's gain came from a memory that pooled evidence across episodes. A key that fragments the same screen pools less.

What it means if your phone agent remembers screens

• Check both directions on your own logs before choosing a merge rule. Here the split rate (60 keys for 29 screens) was the bigger problem, and the over-merges were 4 of 38 and confined to one kind of row.
• Make the key invariant to scroll offset for list rows: keep the label, drop the y cell. On these crawls that would collapse most of the 14 Settings keys. The risk is merging a list with its own detail view when both show the same labels, which the action-conditioned test catches.
• Record behaviour per control, and mark a control unstable when its outcomes disagree. phoneshell's ControlMemory.led_to already counts destinations. best_outcome() reports the most frequent one as if it were certain, so a banner seen twice two ways prints a confident route.
• Give promotional and account-state rows no memory and no exploration budget, as SAGE does. On a personal phone they are the rows most likely to change between Tuesday and Thursday.
• Withholding the screenshot on a "known" screen should depend on the key being stable as well as on two visits.

Check it yourself

# the key and how memory uses it
sed -n '/def fingerprint/,/hexdigest/p' phoneshell/memory.py
grep -n "min_visits_to_trust\|skip_image_on_known" phoneshell/actions.py phoneshell/config.py
grep -n "structural = fingerprint" phoneshell/crawl.py

# crawl your own phone, then group edges
bin/phoneshell collect Settings      # writes runtime/crawls/<stamp>-settings/manifest.json
# for each edge: key = (screens[frm].structural, edge.key)
#                behaviour = 'no-op' | 'left app' | (screens[to].bundle_id, screens[to].title)
# a pair disagrees when its edges carry more than one behaviour

Our eleven manifests are not published. They carry the phone owner's account name and network name on the Settings root. The counts above come from a 40-line script over screens and edges in each manifest, using the fields named in the comment. The papers' numbers are in Table 1 of arXiv:2609.38778 and Section 1 of arXiv:2609.39547.

What would prove this wrong

By 31 March 2027 we will rerun the Settings and Talika crawls on the same phone and compare them with the 11 September set. Two predictions. A scroll-invariant fingerprint (list rows keyed by label without the y cell) will give the Settings screen at most 4 keys instead of 14, without merging any (fingerprint, control) pair whose behaviours disagree. At least 3 of today's 4 disagreeing rows will still disagree or will have vanished, because they are promotions and account banners. If the scroll-invariant key creates new disagreements outside promotional rows, appearance was carrying information we have thrown away, and the bisimulation test belongs in the crawler rather than in a tidier key.

Sources

• Hongbo Zhang, Liuyang Song, Quanquan Li, Daqian Yang, Yan Wen, Zhengtao Yao, "Action Conditioned Bisimulation For GUI Agent Memory", arXiv:2609.38778, 30 September 2026.
• Bo Han, Qianyi Wang, Shuai Liu and others, "Learning Reliable GUI Agents under Imperfect Priors", arXiv:2609.39547, 30 September 2026.
• Blolabel, phoneshell: memory.py, crawl.py, actions.py, read 7 October 2026; crawl manifests of 11 September 2026 on our rig.
• Blolabel, "phone memory keeps what the screen said" and "imagined rollouts thin out by step ten", earlier notes on the same memory store and crawls.