Ten Feature Ideas — State-of-the-Art Feasibility Dossier¶

Date: 2026-06-14 · Tier: design/ — public engineering research Companion: design/v2-cognitive-layer/ · the killer-features 10x study

A feasibility review of ten candidate feature ideas: per-feature offline/on-device feasibility as of 2026, "why now," and evidence grading, gathered from a live web sweep across four parallel research streams. Tags mark evidence tier (tier 1 = peer-reviewed/replicated, descending to tier 7 = forum discussion) and grade A–F for argument quality. A number is reported with the conditions it was measured under; a number without them is treated as a rumor, not a fact.

Verdict table¶

Per-idea findings¶

1. Prosody Ink — partial¶

• FullStop / deepmultilingualpunctuation (XLM-R-large): text-only punctuation restoration, CPU-runnable; EN F1 of 0.948 (period), 0.890 (question), 0.819 (comma) on Europarl [tier 1, grade A]. Sets the text-only baseline a prosody layer would need to beat.
• A lightweight on-device punctuation+casing model (CNN+BiLSTM) runs at roughly 1/40^th the size of a comparable Transformer, 2.5× faster, with a +9% relative F1 gain on IWSLT2011 [tier 1, grade A]. Text-only punctuation already fits a CPU real-time budget.
• Pitch-accent detection improves ASR: word-level prominence F1 of 0.90 on the BURNC corpus with a wav2vec2-base model [tier 1, grade A]. Supports a stress-to-bold mapping; the model was trained on 3×V100 GPUs with no reported CPU/latency figures, on clean broadcast news audio.
• A CNN over 5 openSMILE acoustic features (F0, RMS, loudness, voicing, HNR) reaches 87.5–88.7% pitch-accent classification accuracy; an unsupervised k-means baseline reaches 84%/0.86 F1 [tier 1, grade B]. This is the realistic offline path — a small feature set, faster than real-time on CPU via parselmouth/openSMILE.
• Rising-pitch-to-question-mark mapping is the weak link: WH-questions typically carry falling F0, and pitch-contour-only classification reaches only ~64.6% [tier 2, grade B]. Pause-to-paragraph and stress-to-bold mappings are considerably safer bets than pitch-to-question.
• Verdict: ship pause→paragraph and prominence→bold now; treat pitch→question as experimental.

2. Glance-Type — too-early for caret precision¶

• L2CS-Net reports 3.92° angular error on MPIIGaze, using plain RGB input and available as a pip-installable package [tier 2, grade A].
• The best surveyed webcam-to-screen accuracy is ~3.2–3.3° / ~50 mm RMSE with a still head, degrading to 5.1° / 80 mm with head motion; dedicated Tobii hardware reaches 15 mm/0.9° for comparison [tier 2, grade A].
• GazeRecorder reports ~1.75 cm accuracy but needs ~30 calibration points and loses ~9% of samples under motion; WebGazer reports ~4.06 cm [tier 2, grade B].
• MediaPipe's Iris landmarks run at 90+ FPS on ~5% CPU but do not themselves output a gaze vector — a regressor has to be trained on top [tier 3, grade B].
• WebGazer runs fully client-side and self-calibrates from clicks, but is validated only for coarse, wide screen regions [tier ⅔, grade B].
• Verdict: real, but accuracy-bound — best framed as "look-to-pane" or "jump-to-field" targeting, not caret-precise positioning.

3. Punch-In — partial¶

• Multimodal error correction studies: respeaking corrects only 35% of errors, versus 87% for retyping and 82% for spelling it out, at 2.7 vs. 6.6 WPM; accuracy drops further on a repeated respeak attempt [tier 2, grade A].
• A separate study reports respeaking "often does not lead to correct recognition," with multimodal correction running roughly 2× faster than unimodal [tier 2, grade B+].
• FastCorrect uses edit-distance token alignment to locate and splice corrected spans, reporting −8–14% WER at 6–9× the speed of a full re-decode [tier 3, grade A]. The splice mechanism itself needs no additional training.
• Phonetically-aware alignment can match a respoken span even when the words differ from the original [tier 3, grade B+].
• No product surveyed as of 2025–26 ships in-voice span correction; Dragon's "Correct " plus an alternatives list remains the most mature pattern found [tier 5, grade C].
• Verdict: ship respeak → 2–3 aligned candidates (reranked by a small local model) → explicit confirm, with a keyboard fallback on low confidence. Do not auto-splice without confirmation.

4. Read-Back Loop — ready-now¶

• Kokoro-82M runs CPU-only (measured on a 4-core EPYC) at RTF 0.469 (PyTorch) / 0.509 (ONNX), 24 kHz, human-like quality, Apache-2.0 licensed; latency scales with utterance length, which chunking addresses [tier 6, grade B].
• KittenTTS Nano (15M params, ~25 MB int8) and Mini (80M) are Apache-2.0 and run on a Raspberry Pi/phone/WASM, at lower quality than Kokoro and with no official RTF figure published [tier 5, grade B].
• Piper (VITS-based) reports CPU-first RTF ≈0.2 (~5× real-time); its original MIT-licensed repository was archived in October 2025, and the actively maintained fork since is GPL-3.0 [tier 6, grade C] — a licensing consideration for anyone adopting it.
• MeloTTS (MIT) is explicitly designed for CPU real-time use; Coqui/XTTS carries a more restrictive weights license and is not CPU-real-time [tier 6, grade C].
• Streaming time-to-first-audio in the 50–200 ms range is reachable, and human turn-taking tolerance sits around 200–300 ms — the metric to optimize is time-to-first-audio, not full-utterance RTF [tier 2, grade B].
• Verdict: ready to build. Kokoro or MeloTTS (Apache/MIT) on CPU, with sentence-level chunking for sub-second first audio. Avoid GPL- or restrictive-license alternatives.

5. Voiceprint Mind (personal voice adaptation) — ready-now (continuous learning = partial)¶

• LoRA fine-tuning of Whisper-large needs <8 GB VRAM and roughly 6–8 hours on a 12-hour personal corpus, producing an adapter of ~60 MB (<1% of model parameters) [tier 4, grade C].
• On-device personalization measured a −44% relative WER reduction (25.11%→17.7%) on atypical speech, with gains compounding across sessions [tier 2, grade B] — a direct evidence basis for a "personalization compounds locally over time" design.
• Residual/PEFT adapters can personalize at <0.5% of parameters per speaker, with continual adaptation and no measured forgetting of the base model [tier 2, grade B].
• Hot-word biasing helps but is not free: rare-word WER improves from 23.7% to 18.0% and out-of-vocabulary recall from 60% to 37.1% miss rate, but error can rise on words not in the bias list [tier 2, grade B].
• TCPGen neural biasing (the open-source WhisperBiasing project) improves user- vocabulary recall over a fixed 1000-word list approach [tier 2, grade B].
• Error-driven data selection (using corrections a user has already made, rather than random sampling) outperforms random selection for fine-tuning data [tier 2, grade B].
• Verdict: ship (a) prompt/vocabulary biasing first, since it is the cheaper and faster-to-ship mechanism, then (b) an opt-in, scheduled background fine-tune from stored correction history — "continuous" here means periodic background re-tuning, not live in-session learning.

6. Cocktail Filter (target-speaker gate) — ready-now (gate); partial (full separation)¶

• VoiceFilter-Lite: 2.2 MB, 8-bit, streaming, real-time on-device, reporting −25.1% WER in overlapping speech and −14.7% in reverberant conditions, with no measured harm to clean speech [tier ⅔, grade A] — a close match to the target capability, already in production elsewhere.
• Personal VAD models run at ~130K parameters, speaker-conditioned, producing a per-frame target/non-target decision [tier 2, grade A].
• Silero VAD runs at ~1–2 MB with <1 ms CPU cost per 30 ms chunk (RTF 0.004) [tier 3, grade B].
• TargetVoice (Interspeech 2025) is a low-latency streaming target-speaker extraction engine designed for CPU [tier 2, grade B].
• Short-enrollment d-vector speaker embeddings are the current standard, with the 2025–26 trend moving toward even shorter, or enrollment-less, embeddings [tier 2, grade B].
• SepFormer reaches a strong 19.4 dB SI-SDRi but is too heavy for CPU real-time — full source separation remains a GPU-tier capability [tier 2, grade A].
• Verdict: ship enroll-once, then a personal-VAD-plus-Silero gate with VoiceFilter-Lite-style suppression — not full source separation.

7. Ghost Ahead (predictive text) — too-early; pivoted¶

• Predictive text does not reliably speed up entry in controlled studies: 33 WPM with prediction vs. 35 WPM with none vs. 43 WPM with autocorrect alone; best-case gain +2 WPM, worst-case −8 WPM [tier 2, grade A].
• Word prediction suggestions are accepted for only ~1.6% of words typed [tier 3/2, grade B].
• GitHub Copilot's ghost-text completions are accepted 26–35% of the time — the strongest precedent found — but that is for code, and roughly two-thirds of suggestions are still ignored [tier 2, grade A].
• Upcoming-speech prediction has been studied specifically as end-of-turn timing rather than content: a 25M-parameter model forecasts turn-end up to 2.56 s ahead, cutting measured response latency from 1195 ms to 690 ms [tier 3, grade B+].
• Sub-1B-parameter GGUF models are fast enough (<200 ms for 3–5 words) but weak at open-ended continuation [tier 4, grade C].
• Verdict: content prediction is not demonstrated to work — pivot to endpoint/turn anticipation (hiding latency around when the user is about to stop speaking), which has real supporting evidence.

8. Mid-Thought Undo (spoken self-correction) — partial¶

• "Teaching BERT to Wait": streaming disfluency detection at ~3.1M parameters (~35× smaller than comparable models, ~80% latency reduction), reporting state-of-the-art streaming F1 on Switchboard [tier 1, grade A].
• "Toward Interactive Dictation" (TERTiUS): open-ended spoken edits, where a small targeted model reaches 30% success at 1.3 s versus a large LLM's 55% success at 7 s [tier 1, grade A] — the closest prior art found, and evidence that open-ended reformulation is a genuinely hard problem.
• Reparandum (self-correction span) removal on Switchboard reaches ~0.93 F1 but degrades in streaming and out-of-domain settings [tier 2, grade B].
• A 1–3B-parameter Q4-quantized local model can parse a fixed pattern like "scratch that, make it X" in under 1 second on CPU [tier 5, grade B].
• Dragon, macOS, and Talon all ship a "Scratch That" command, but as a fixed template acting on the last utterance only — not open-ended reformulation [tier 6, grade C].
• Verdict: ship the fixed "scratch that"/last-burst-delete template now; gate arbitrary "no, make it X" rewrites behind a confidence threshold and an undo path.

9. Polyglot Switch (mid-sentence language switching) — partial (one configured pair)¶

• Stock Whisper cannot code-switch within an utterance — one paper reports it "failed to produce any code-switched words" over a 30-second window [tier 2/7, grade B/C].
• Code-switched speech raises WER by 30–50% relative to monolingual speech in reported benchmarks [tier 5, grade C].
• An adapted small Whisper model reaches 14.0% MER on Mandarin-English code-switching (SEAME, 101 hours) via encoder refinement and language-aware decoding, against a 58.3% baseline [tier 2, grade B] — a small enough model to be CPU-plausible.
• A concatenated-tokenizer approach reaches 98%+ per-span language identification accuracy [tier 2, grade B].
• Soft-prompt PEFT adaptation for code-switching is reported to compete with full fine-tuning [tier 2, grade B; table figures not independently re-verified].
• Verdict: buildable offline for one explicitly configured language pair at a time — not an arbitrary-language "any blend" capability. Requires an adapter and code-switched training data (e.g. SEAME or SwitchLingua).

10. Say-Macro (voice macros) — ready-now¶

• Voice macros are the highest-retention feature in clinical speech-recognition software: 91% usage and 72% "very helpful" rating among continuing users [tier 2, grade A].
• On-device speech-to-intent on a fixed grammar reaches >99% accuracy in clean conditions and 97% at 9 dB SNR, on a 619-command set (Picovoice Rhino benchmark) [tier 4, grade B].
• voice2json performs offline grammar-to-{intent, slots} parsing with retraining in under 5 seconds [tier 4, grade B].
• Talon, Caster, and Numen all ship trigger-to-snippet or trigger-to-OS-action macros offline, though configuration-heavy [tier 7, grade B].
• Free-form speech recognition reports errors in ~22% of dictations in one study; macro expansion sidesteps misrecognition risk for the covered phrases [tier 2, grade B].
• Verdict: ready to build — the substrate (a grammar classifier feeding a dispatcher) already exists; the remaining work is misfire suppression via full-utterance context gating and an explicit macro mode.

Timing synthesis¶

The decisive 2024–2026 shifts that move several of these from "too-early" to "ready": (1) Apache/MIT-licensed neural TTS reaching CPU real-time (Kokoro-82M) — unlocks Read-Back Loop; (2) PEFT/LoRA collapsing personal fine-tuning to an overnight laptop job, with on-device −44% WER results — unlocks Voiceprint Mind; (3) 2.2 MB quantized streaming target-speaker models already in production — unlocks Cocktail Filter; (4) grammar-constrained intent classification at 97%+ on-device — unlocks Say-Macro. Two ideas remain genuinely field-gapped: free-form spoken content prediction (Ghost Ahead) and cursor-grade uncalibrated webcam gaze (Glance-Type) — both need a pivot or a narrower scope rather than more engineering.

Evidence-ranked build order at the time of this study¶

Tier A — highest evidence, ready to build on existing infrastructure 1. Cocktail Filter (gate) — reuses voice-enrollment infrastructure; strong production precedent; trivial CPU cost. 2. Voiceprint Mind — biasing layer first, nightly fine-tuning second; extends the existing local learning loop. 3. Say-Macro — the grammar/dispatch substrate already existed; only misfire-gating remained. 4. Read-Back Loop — a clean accessibility win, unlocked by CPU-real-time TTS.

Tier B — ship the well-evidenced sub-feature, gate the harder one 5. Prosody Ink → pause→paragraph and stress→bold now; pitch→question experimental. 6. Mid-Thought Undo → fixed "scratch that"/last-burst delete now; open-ended rewrites gated. 7. Punch-In → respeak→candidates→confirm, with a keyboard fallback. 8. Polyglot Switch → one opt-in, explicitly configured language pair.

Tier C — re-scope before building 9. Glance-Type → "look-to-pane" coarse targeting only, not caret precision. 10. Ghost Ahead → pivot to endpoint/turn anticipation; drop content prediction.

What shipped¶

See design/v2-cognitive-layer/ and design/adr/ for which of these ideas were built, and in what form — several of the Tier A and Tier B items above (Cocktail Filter, Voiceprint Mind personalization, Read-Back Loop, Say-Macro, Polyglot Switch, Glance-Type as gaze routing, Punch-In) now have shipped, tested implementations and their own ADRs.