iOS Settings exposes no selected state, so a check that finds the word Never passes whether or not Auto-Lock is Never

Three checks reported success on a phone that was not set up: a selection check that looked for text, a Settings walk that never left its first screen, and an app launch that opened a system service. Each fix replaces a presence check with a witness.

On iOS 26.6 the Auto-Lock page in Settings exposes no selection state in the accessibility tree: no selected trait, no value, no isSelected. Our rig confirmed it had set Auto-Lock to Never by finding the word "Never" on the page, a check that passes whether or not Never is chosen, and the phone kept locking mid-run for four benchmark runs. The only evidence of the chosen row is a Button labelled checkmark inside it, and the tree-condensing step we feed to models was dropping that button as decoration.

Three checks that passed on a phone that was not set up

All three were found on 8 September 2026 by running the benchmark unattended on a physical iPhone 17 Pro Max, and each is a check that reports success because it looked at the wrong thing. They are recorded in sections 10 and 22 of our findings file and fixed in phoneshell/prep.py and phoneshell/apps.py.

The first is the selection check above. Apple documents a selected accessibility trait ("the accessibility element is currently in a selected state") and an isSelected attribute on XCUITest elements. Neither is set on the Auto-Lock rows. The chosen row differs from the others in exactly one way: it has a child Button whose label is checkmark.

The second is the walk to that page. A tap on a Settings row does not always take. When it did not, the walk carried on searching the screen it never left, then reported the child row missing: "could not find Auto-Lock in Settings", on a phone where Auto-Lock is one swipe below Display & Brightness. The error names the wrong failure, and the silent version of it is the one that let the phone lock during a run.

The third happens before any of that. open_app("Contacts") resolved to com.apple.PeopleViewService, a system service that shares the display name, instead of the Contacts app at com.apple.MobileAddressBook. The task failed with the model having done nothing wrong. We now pin 41 spoken names to bundle ids; the map was first checked against the phone's own app catalogue on 7 September 2026, and Contacts was added after this failure.

What each check looked at, and what was true

The common shape: each broken check asked whether something was present, and the fix asks whether something changed or is attached to the right parent. Presence is the cheap question, and on iOS it is the one that lies most often.

Setting one system option, done so that it proves itself

Our prep step is written as a general routine, "walk Settings down a path and choose an option", because every system setting a rig depends on has the same shape. Each step carries its own witness.

What it means for anyone grading or driving a phone agent

A task checker that confirms a setting by finding the option's text is measuring whether the option exists. If your benchmark has any "set X to Y" task on an iOS single-select list, check how it is graded. The same blind spot applies to the model: in our condensed tree the model could not see which row was chosen either, so an agent asked "is Auto-Lock set to Never?" had no way to answer from the tree. We now keep the checkmark in the raw read used for verification; whether to surface it to the model is a separate decision we have not measured.

Resolve apps by bundle id, never by display name. Several iOS bundles share a display name, and some are services rather than apps. A curated map is tedious and it is the only thing that removed this class of failure for us.

Do not trust a phone-side setting the rig did not set and verify itself. Auto-Lock had been set to Never by hand on this phone and it still locked mid-run. A lock during an unattended run is expensive, because every task after it fails identically, and a reboot to recover from it ends the session (see the note on reboots).

A judgement about where this is heading: Apple's WWDC26 session on App Intents and onscreen awareness (8 June 2026) is a route by which an app can describe its own state to the system in structured form. If Settings ever exposed its options that way, the checkmark heuristic would be unnecessary. Nothing we have seen on iOS 26.6 does so for the Auto-Lock page.

Check it yourself

Open Settings > Display & Brightness > Auto-Lock on a phone running WebDriverAgent, then read the tree. Every attribute the tree has for the rows will be on these lines.

# the raw XML for the page: compare the chosen row to its neighbours
curl -s localhost:8100/source | grep -i -E 'Seconds|Minute|Never|checkmark'

# the same as JSON, cells and the checkmark with their frames
curl -s 'localhost:8100/source?format=json' \
  | jq -r '.. | objects | select(.type=="XCUIElementTypeCell" or .label=="checkmark")
           | [.type, .label, .value, .rect.y, .rect.height] | @tsv'

# change the selection by hand and read again: only the checkmark's y moves

# our routine and its witness
grep -n "def _row_is_ticked" -A 24 phoneshell/prep.py
grep -n "def choose_setting" -A 40 phoneshell/prep.py
grep -n "PeopleViewService" -B 4 -A 4 phoneshell/apps.py
sed -n '/^## 10\./,/^---/p' FINDINGS.md

What would prove this wrong

• Read the Auto-Lock page on iOS 26.7 or iOS 27 and find the chosen row carrying the selected trait, a value, or isSelected true through XCUITest. That would make the checkmark heuristic a workaround for one version. We predict it is still absent on every iOS 27.x release through 30 June 2027.
• Switch the phone to a non-English locale and find no button labelled checkmark. We have only tested English. If the label is localised, our witness fails on those phones, and it fails as "setting not applied" rather than silently passing, which is the correct direction to fail.
• Find another single-select page in iOS Settings that does expose selection. That would show the gap is per-page rather than systemic, and a checker could prefer the attribute where it exists.