A 35B model now runs on an iPhone 17 Pro Max at 1.8 to 3.1 tokens a second, too slow to read one agent turn

Routide pages a Qwen3.6-35B-A3B checkpoint's experts from storage and reports everything, including the runs that went wrong. Memory fits in under 3 GiB. Prompts are read one token per forward, so a phone agent's 2,542-token tool list alone would take 14 to 24 minutes.

A 35-billion-parameter mixture-of-experts model now runs on an iPhone 17 Pro Max by paging its experts from storage, decoding at 1.8 to 3.1 tokens a second. The runtime, Routide, processes prompts one token per forward pass. At those rates, the 2,542 tokens of tool definitions that phoneshell sends an agent on every turn would take 14 to 24 minutes to read, by our extrapolation, before the model saw a single screen. The paper is arXiv:2609.29032, submitted 24 September 2026 by Musa Shams.

What Routide showed, on which phone

Routide is a Swift and MLX runtime for one pinned public checkpoint, mlx-community/Qwen3.6-35B-A3B-4bit at revision 38740b8. The text path has 40 layers, 256 routed experts per layer and top-8 routing, so each forward pass requests 320 experts, 540 MiB of expert payload if every one misses. All routed experts together are 18,119,393,280 bytes; one expert is 1,769,472 bytes (1.6875 MiB). Experts stay in storage and a byte-budgeted cache holds a subset in memory. The phone is an iPhone 17 Pro Max, measured on iOS 26.5.2, 26.6.1 and 27.0 across cohorts.

The paper is careful to a degree that is rare. It keeps a run that hit a thermal stop, reports timing comparisons that went the wrong way, separates logical reads from physical storage traffic, and states what it does not claim: not the first phone LLM runtime, not the fastest throughput, not quality parity with a resident model. Its outputs from the phone and from a resident Python reference disagree on all five tested prompts, so it makes no numerical-equivalence claim either.

The 512 MiB "cliff" is a cache policy, not a memory size

At a 512 MiB expert budget, LRU eviction replayed over the recorded routes scored 0.00% demand hits: every expert it kept was evicted just before it was needed again. Our arithmetic suggests why: 512 MiB holds 303 expert payloads of 1.6875 MiB, and one forward cycles through up to 320 distinct ones (540 MiB). A loop larger than the cache is the textbook case where least-recently-used evicts each item just before its reuse. Random eviction at the same budget hit 18.80%. Adding 64 MiB lifted LRU to 38.58% and cut cold logical reads from 16.987 GB to 9.709 GB, 43%; the next 64 MiB changed nothing. The paper's conclusion, which we agree with, is that the apparent capacity cliff is a policy and workload interaction. Anyone sizing a phone runtime from one LRU sweep would draw the wrong line.

Speed is not stable even on nominal thermals. Two runs of the same configuration differed by 10.58% in elapsed time, and one of six paired 128-token comparisons ran 7.18% slower at the larger cache despite 38.16% fewer reads. A 512-token generation read about 320 GB of logical expert payload, roughly 0.62 GB per token by our division. That is bytes delivered by the reader, not NAND traffic, and the paper says it is not a wear measurement.

What it means for an agent running on the phone

The number that decides whether a local model can drive a phone is not decode speed, it is how long the model takes to read a turn's input. Routide reads prompts one token per forward and has not implemented batched prefill; the longest prompt it ran was 340 tokens. An agent turn is larger than that before any screen is attached. phoneshell's MCP tool definitions alone are about 2,542 tokens, sent on every turn. At the one-forward cost implied by the measured decode rates, 0.33 to 0.56 seconds, that is 13.9 to 23.6 minutes. The same agent on a cloud model measured 4.1 to 6.8 seconds per whole turn on our rig. The extrapolation assumes prefill forwards cost what decode forwards cost, which the paper does not measure, and it would shrink with prefix caching, which pays the static tool list once per session. It would not shrink for the screen, which changes every turn.

The memory side is more encouraging. A 35B-total model held sampled process footprints of 1.87 to 2.73 GiB on this phone, and ran 128-token generations to completion without a memory warning. For an agent the practical design today is split: a small resident model on the phone for short, frequent decisions, and the large model elsewhere. A 35B-class local planner is feasible as a memory claim and not yet as a latency claim.

If you benchmark a runtime like this, copy the paper's habits. Keep stopped runs. Report paired comparisons that went the wrong way. Separate logical reads from storage traffic. Its 10.58% spread between identical runs is the noise floor any claimed speedup on this phone has to clear.

Check it yourself

• Paper: https://arxiv.org/abs/2609.29032; HTML https://arxiv.org/html/2609.29032v1. Section 3.1 (checkpoint inventory), 3.2 (serial prefill), 5.1 (cache budgets), 5.4 (sustained runs), 5.5 (memory).
• Code: https://github.com/MusaShams/Routide. Checkpoint: https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit, pinned revision 38740b847e4cb78f352aba30aa41c76e08e6eb46.
• Our arithmetic: 2542/3.052/60 is 13.9 minutes and 2542/1.795/60 is 23.6; 319.354/512 is 0.62 GB per token; 1 - 9.709/16.987 is 0.43.
• phoneshell: FINDINGS.md section 8 (2,542 tokens of tool definitions per turn) and section 9 (4.1 to 6.8 seconds per turn).

What would prove this wrong

Two predictions, dated 31 March 2027. First, no public measurement will show a flash-paged model of 30B or more total parameters on an iPhone reading a prompt of 2,000 tokens or more in under 60 seconds. A result like that, from Routide with batched prefill or from anyone else, would make an on-device planner for phone agents a latency question worth reopening, and our extrapolation obsolete. Second, a measured Routide prefill of a 2,542-token prompt, if one is published, will take longer than 10 minutes on an iPhone 17 Pro Max with the serial path; under 10 minutes would mean prefill forwards are much cheaper than decode forwards and our extrapolation overstated the cost.