Android on-device ASR: device-tier measurement protocol and low-memory model policy¶

Date: 2026-11-06 Status: Research note and protocol design (not a decision). Synthetic and unrun: no device was used, no APK was built, no Gradle task or Android test was run, no model was downloaded, and this note contains no measurement. ADR-030 freeze unchanged. No ADR is edited or accepted by this note. Tracking: NEXT-MOBILE-03 #808, parent #789; implementation epic #81; feeds #92 (:bench harness and device matrix) and #87 (:core:stt + :native:whispercpp); related spike #97 (90-minute microphone FGS). Repository snapshot: main at 4695e49a. Builds on, and does not repeat: the pinned offline ASR evaluation protocol (metric definitions, statistics, result contract), the small and quantized ASR models note (candidate grid C1-C9), the local accelerator backends note (resolved-backend attestation, energy rule), the runtime and model provenance snapshot (licences and the per-run provenance record), the Android API-level and OEM IME note (constraints C1-C20 and device-test checklist T1-T12), and Proposed ADR-033. Binding mobile decisions: ADR-MOB-005, ADR-MOB-006.

Labels: [Verified-in-code] read in this repository at the snapshot; [Verified-in-doc] stated by a repository document (ADR or research note) that is cited; [Assumption] my inference, named so it can be struck; [Unknown] not established by anything read. Android platform APIs named in sections 3-5 as candidate measurement methods were not re-read against developer.android.com for this note; each is an [Assumption] until #92 confirms it on a device.

1. Answer to the issue question¶

Question (#808): which pinned whisper.cpp / sherpa / Parakeet or quantized configurations merit tests on budget phones after M1 works?

• Only two configurations are pinned today, and they are the first matrix. The in-APK catalogue holds exactly two entries, both for the whisper.cpp engine, both at Hugging Face commit 5359861c739e955e79d9a303bcbc70fb988958b1 of ggerganov/whisper.cpp [Verified-in-code, models.json]:

The MIT licence is the catalogue's recorded value; the small-quantized note records the same files' Hugging Face card tag as mit (row C1). No licence determination is made here. - Every other candidate is unpinned and cannot enter the matrix yet. Quantized ggml Whisper files (small-quantized note row C1), sherpa-onnx int8 families (C7), Moonshine streaming checkpoints (C6) and the parakeet.cpp GGUF ladder (C9) have no immutable revision and file SHA-256 in this repository. The small-quantized note section 5 already says no row has been resolved to a revision [Verified-in-doc]. Under ADR-MOB-006 decision 7 a model addition is "a reviewable PR with a checksum in the diff" [Verified-in-doc], so a candidate enters this protocol only after it has a catalogue row. - Licence gate before any arm: Parakeet TDT v2/v3 weights are CC-BY-4.0, Nemotron 3.5 is OpenMDW-1.1 and needs review, nemotron-speech-streaming is under the NVIDIA Open Model License, and Moonshine legacy non-English checkpoints are non-commercial; all four rows are from the provenance snapshot section 3 [Verified-in-doc]. A non-commercial or unreviewed row is not a candidate. - Recommendation (for the owner, not a decision): first device run = tiny.en and base.en at the pinned commit, one engine, across the tiers in section 3. Quantized and second-engine arms wait for (a) a pinned catalogue row and (b) the schema additions in section 4. This keeps to #808's "avoid a mandatory second engine" constraint.

2. What the Android tree has today (so the protocol targets real seams)¶

Consequence. The protocol below is runnable only after #87 (an engine behind the seam) and #92 (a harness in :bench) land. Until then every number in the matrix is [Unknown], and the doc and app state performance as "measure it on your device" (ADR-MOB-005 decision 7) [Verified-in-doc].

3. Device tiers¶

Tiers are defined by properties the harness reads from the device, never by brand or price, so a report places itself and a reviewer can check the placement.

The RAM bounds are deliberately left blank: no source read gives a per-device model budget (API note Q7 and C14: "unknown numbers") [Verified-in-doc]. [Assumption] The owner sets them after the first T1-T3 reports, using measured peak RSS, not marketing RAM.

Properties to record for tier placement, all candidate APIs [Assumption]: total and available memory and the low-RAM flag from ActivityManager; getMemoryClass() (named in the API note Q7 [Verified-in-doc]); supported ABIs from Build.SUPPORTED_ABIS; SoC name (already DeviceInfo.soc); Android version and patch level, OEM skin version and install source (API note section 7 run record [Verified-in-doc]); page size (4 KB or 16 KB, API note Q8 and C16 [Verified-in-doc]).

ABI rule. A device without arm64-v8a, or a 16 KB-page device given a library not aligned for it, is an unsupported row, not a WER of 100% and not a missing latency (the pinned protocol's rule for a machine that lacks a backend [Verified-in-doc]). Whether 32-bit armeabi-v7a is built at all is undecided in the tree (no abiFilters) [Verified-in-code]; the protocol records it, it does not decide it.

4. Metrics and where each one lives¶

Definitions are the pinned protocol's section 5; this table only maps each #808 metric to an Android method and to a field that does or does not exist in schema /1.

Schema rule. BENCH_SCHEMA's KDoc says to bump the suffix on any incompatible change [Verified-in-code, BenchResult.kt L3]. Gaps G1-G8 therefore belong to a schema /2 under #92, with every new value a Reading so "not measured" stays distinct from zero. This note does not change the schema.

5. Run procedure (per device, per arm)¶

1. Provenance first. Record the per-run provenance list of the provenance snapshot ("Provenance record required for each experimental run"), plus the catalogue id, pinned commit and file SHA-256, and the whisper.cpp source commit of the build. A model whose SHA-256 does not match the catalogue is not run (ADR-MOB-006 decision 3 treats a mismatch as a security event) [Verified-in-doc].
2. Offline. Network access for the app denied before the first clip; ADR-MOB-006 decision 4 already requires the app to work that way [Verified-in-doc]. A run with network allowed is invalid.
3. Corpus. Licensed public or synthetic clips only, from the frozen manifest of #793 and the pinned protocol section 3. No private dictation, no user audio (#808 and AGENTS.md). Include the pinned protocol's silence and long-pause stress slices, since G3 is otherwise unobservable.
4. State recorded, not controlled away: charge state and charger connected or not, screen on, battery-optimisation state, standby bucket (adb shell am get-standby-bucket, API note section 7 [Verified-in-doc]), thermal state at start.
5. Cool-down gate. An arm starts only at the coolest thermal state the device reports [Assumption about the state name]; if that state is not reached within a fixed wait, the arm is recorded as not_started: thermal rather than run hot.
6. Repetitions. At least 3 independent process launches per arm, one recorded warm-up, shuffled clip order, model order randomised per device (pinned protocol section 5). Each run is kept; none is discarded as an outlier.
7. Cold versus warm. "New process, warm page cache" and "first run after reboot" are separate labelled conditions, because a ggml model file is memory-mapped and unmodified mmapped pages can be reclaimed (API note Q7 [Verified-in-doc]).
8. Soak. One sustained arm per device (repeat the corpus for a fixed wall time) records thermal state and per-clip wallS over time, so throttling shows as drift rather than as noise. The 90-minute FGS question stays with #97.
9. Paired comparisons only on the same device. Cross-device rows are descriptive. No leaderboard mixes tiers, normalisers or failed cells (pinned protocol section 6).

6. Low-memory safe fallback policy (design proposal)¶

The failure being designed against: a model too large for the device either fails to load, or loads and is then killed by lmkd mid-burst, which on an IME takes the keyboard with it (API note C14; ADR-MOB-005 notes a native crash takes the IME process) [Verified-in-doc]. Rules, in order:

1. Measured admission, not an assumed budget. Before loading, compare available memory against the measured peak RSS for that catalogue id on the device's tier, from :bench reports. With no measurement for the pair, the app loads and records the outcome; it does not invent a threshold. [Assumption: the margin above measured peak RSS is an owner decision after T0/T1 data]
2. Refuse visibly, never degrade silently. If admission fails, the burst does not start and the user sees a typed reason ("this model needs more free memory than the device has now") with the storage/model screen one tap away. This is the honesty rule the desktop applied to engine fallback in #644 (a recorded engine_fallback in status) [Verified-in-doc, accelerator note section 4].
3. Fallback goes down the catalogue, same engine, with consent. The only automatic suggestion is a smaller model that is already downloaded and verified (for the current catalogue, base.en to tiny.en). The switch needs a user tap and is recorded; it is never silent, and the app says the smaller model is less accurate (ADR-MOB-005 consequence on honest expectations) [Verified-in-doc].
4. No download as a fallback. A missing smaller model is offered for download only through the normal chooser after an explicit tap (ADR-MOB-006 decisions 2-3) [Verified-in-doc].
5. No second engine as a fallback. Falling back from whisper.cpp to sherpa-onnx would make the second engine mandatory, which #808 rules out and ADR-MOB-005 decision 3 keeps optional [Verified-in-doc].
6. No cloud fallback, ever (ADR-MOB-005 Rejected) [Verified-in-doc].
7. Killed mid-decode. If the process dies or the engine reports an allocation failure during a burst, no partial text is committed, the PCM is not persisted, and the next IME start shows that the last burst was lost. [Assumption: how the IME detects its own prior death is a design question for #87]
8. Residency is measured, not decided here. API note C15 records the conflict between releasing large allocations when the IME hides and keeping a model hot; the protocol measures both sides (cold reload latency against resident memory per tier) so the owner can decide with numbers [Verified-in-doc for the conflict].
9. In the bench record, a refusal is low_memory_refused, a lmkd kill is killed, and a consented smaller model is fallback with fallback_from; none of them contributes a WER or latency number (pinned protocol result contract) [Verified-in-doc].

Rules 1-3 and 7 would change product behaviour described by ADR-MOB-005/006. Whether that needs a Proposed amendment is an owner decision; this note proposes no ADR text.

7. Failure and edge cases kept as first-class rows¶

Low-RAM refusal; lmkd kill during decode; model file evicted from page cache between bursts; SHA-256 mismatch; 16 KB-page device with a 4 KB-aligned library; device without arm64-v8a; thermal throttling mid-arm; empty transcript on speech; words emitted on silence; decode longer than the clip (RTF above 1); app network access denied (must still work); battery saver enabled; restricted standby bucket (API note C12-C13 [Verified-in-doc]). Each yields a typed status, never a dropped row.

8. CI-only versus verified-on-device¶

Emulator runs belong in the left column only.

9. Links back to #81 implementation issues¶

• #92 owns the harness, schema /2 (gaps G1-G8) and the community device matrix.
• #87 owns the engine, the allocation-failure error type and the cancellation gap noted in SttEngine.kt.
• #97 owns long-duration FGS survival; this protocol's soak arm (step 8) is shorter and does not answer it.
• The API note section 7 device tests (T1-T12) run on the same devices and should share the run record of section 3. Its T7 (peak RSS and PSS per model, C14/C15) is the same measurement as this protocol's peak-memory row and should be run once, under this protocol's method field, not twice.

10. Owner decisions needed¶

1. RAM bounds for T0-T2 (section 3), after the first reports.
2. Whether armeabi-v7a is a supported ABI at all.
3. The admission margin of section 6 rule 1, and whether rules 1-3 and 7 need a Proposed ADR-MOB amendment.
4. Which quantized or second-engine candidate, if any, gets a pinned catalogue row and so enters the matrix (small-quantized note section 6 options apply).
5. Whether energy is instrumented on any device, or the CPU-seconds proxy is the only battery metric.

11. What would change this protocol¶

• A first T1 report showing base.en refused or killed: tiny.en becomes the T1 default question, and section 6 moves from design to priority.
• A device test showing the IME is not treated as foreground while recording (API note Q4): every hold-to-talk latency arm would need a foreground service, which changes the battery and thermal arms.
• A pinned quantized ggml row with a measured lower peak memory at a measured WER cost: it becomes the first arm after the two current models.

Status line. Protocol design only: synthetic and unrun. No device, emulator, Gradle build, Android test or model download was used. No number in this note is a YazSes measurement; every performance value it would produce is [Unknown] until #87 and #92 land and a named device is run.