Screenshotting every screen of an iPhone needs no model, it needs 80 labels it will not tap

The expensive part of an agent step is the decision, not the read: 1 to 3 seconds for the model against 159 ms for the tree. Delete the decision and a screen collection becomes a budget of time.

Collecting a screenshot of every screen in an iPhone app is a graph traversal, and a language model is the worst available tool for it: it forgets which of eleven rows it already opened and pays a second of latency and a fraction of a cent to rediscover that each time. Our crawler does the same walk in code at about 1.2 seconds a step. The part that makes it safe to point at someone's personal phone is not the traversal. It is an 80-label refusal list and five type rules that decide what may be tapped at all.

The refusal list is the product

A crawler tapping its way through Settings, Mail and a banking app will eventually reach a control that spends money, deletes data, places a call or grants a permission, and it cannot undo any of them. So the tap decision is a series of refusals before it is a traversal. Counted from phoneshell/crawl.py on 20 September 2026:

Two details in that list took a real run to learn. Matching is by whole label as a word, not as a substring, so "Delete" is refused and "Deleted Items" stays a screen worth having. And "dictate" is on the never-tap list because the Settings search bar's dictation button sent a crawl straight to the home screen on its very first tap: a microphone control starts capturing and can background the app.

Two keys, because a list of 50 contacts is one screen

The question that decides whether a crawl finishes is "have I seen this screen before", and it has two different answers. memory.fingerprint hashes the layout and ignores the contents. tree.screen_signature hashes the contents: every element as type:text[:30]:round(y/20), sorted, SHA-1, first 16 hex characters.

A new signature is worth a screenshot, because collecting is cheap. Only the first three screens per layout are worth descending into, because exploring is not. That single rule, variant_cap = 3, is the difference between a crawl that finishes and a crawl that walks into 50 contact detail screens and reads out someone's address book.

Fingerprints have to be defended. iOS bakes instance data into accessibility identifiers, including UUIDs, MAC addresses and raw pointers, which survive one launch and change on the next. Unstripped, every revisit looks like a screen the crawler has never seen, and the crawl never terminates. The volatile-text rule does the same job for prices, timestamps and distances.

Breadth, budgets, and the sign-in sheet

The crawl is breadth-first, and the reason is what the collection is for. A depth-first crawl of Settings measured here spent its entire budget inside the Apple Account sign-in sheet: ten screens, nine of them the same modal, and it never reached General, Display or Privacy. Someone asking for "every screen in this app" wants the app's shape first and the third level of one branch last, which is the order a queue gives you and a stack does not. The price is navigation, because a queue sometimes has to replay a path from the root.

Live content gets its own branch, and it was expensive to learn. A screen whose contents tick on their own, a progress ring counting up every second, produces a new content signature on every read. Recording that as a new screen and then relaunching the app to get "back" to the screen the crawl never left cost 12 relaunches in two minutes of one scan. The fix is to compare against the screen's own keys before concluding anything moved.

Check it yourself

git clone https://github.com/Blomega/phoneshell && cd phoneshell

# the refusal sets, counted rather than described
python3 - <<'PY'
import re
src = open('phoneshell/crawl.py').read()
for name in ("NEVER_TAP","CAREFUL_TAP","BACK_CONTROLS","MUTATING_TYPES",
             "INPUT_TYPES","DENY_BUNDLES","APPLE_USER_APPS"):
    body = re.search(name + r"\s*=\s*[\(\{]((?:.|\n)*?)[\)\}]\n", src).group(1)
    print(name, len(re.findall(r'"[^"]*"', body)))
PY

# the two identity keys
grep -n "def screen_signature" -A 4 phoneshell/perception/tree.py
grep -n "_ID_NOISE\|_VOLATILE" -A 6 phoneshell/memory.py

# the liveness threshold and why it is 14
grep -n "_dead_taps >= 14" -B 4 phoneshell/crawl.py

Asking a real iPhone what is installed returns 428 bundles, and roughly 300 of them are invisible UI services such as AXUIViewService and BusinessActionSheet that exist to draw one sheet inside another app. They cannot be launched to a screen. The crawler carries 44 Apple bundle ids it treats as real apps and 13 name shapes it treats as never-an-app, which is a maintenance cost we accept rather than a rule we derived.

What would prove this wrong

The claim with the most to lose is that a label list is a sufficient safety rail. It is falsifiable by a single counterexample: one control on a first-party iOS app whose label is outside all 80 entries and whose tap is irreversible. We expect to find one, which is why every skipped control is logged with its reason. If a full-phone crawl of the 44 recognised Apple apps completes by 31 December 2026 with no irreversible action and no manual repair, the rail held for first-party apps. Third-party apps are a separate claim and this note does not make it.