Skip to content

Wave D — feature research (SoA sweep, 2026)

Date: 2026-07-02 · Tier: design/ — public engineering research · Author: Mohsen Seyedkazemi Ardebili Companion: fed adr-v2-014 through adr-v2-024. See the waves index.

A snapshot of the field on the date above, kept as the research record behind the ADRs it fed. Feature status (shipped, dormant, or design-only) should be checked against yazses features and the linked ADRs, not against this note.

Distinct from the 13 v2 features designed before this wave; all candidates respect the project's core invariants (on-device, zero telemetry, off by default, dependency-isolated). Ranked strongest-first.

Build tiers

  • Ship-now (buildability × value): #3 denoise, #2 X→English translate, #4 predictive completion, #6 tone formatting.
  • Medium: #1 meeting scribe, #5 voice-guard, #8 RAG, #7 codec streaming.
  • Research: #9 silent-speech, #10 VLM commanding, #11 atypical LoRA.

Features

  1. Ambient Meeting Scribe — on-device streaming diarization ("who said what") + summary. Anchor: NVIDIA Streaming Sortformer (arXiv 2507.18446); reuse enrolled voiceprint to tag "you". Extra [scribe]. Medium. Caveat: CPU real-time on ≥3 speakers needs a lighter clustering fallback. → adr-v2-019.

  2. Real-time offline speech translation — dictate in L1, inject L2. Anchor: Whisper task=translate (X→English, zero new deps) → Meta SeamlessM4T v2 (arXiv 2312.05187) for N-to-M. Extra [translate]. Easy (X→En) / Hard (Seamless). Caveat: Seamless heavy on CPU — ship X→English first. → adr-v2-014.

  3. Real-time noise-suppression front-end — denoise/dereverb before STT. Anchor: DeepFilterNet3 (112K–3.58M params, live plugin). Extra [denoise]. Easy. Attacks the "Silent audio — discarding" failure mode. Caveat: over-suppression clips soft consonants → graded on/off + re-run mic-level. → adr-v2-015.

  4. Predictive dictation completion — tiny on-device LLM proposes sentence end; accept by voice. Anchor: Gemma 3 270M (INT4 ~125MB) / SmolLM3, fed corpus n-grams. Extra [predict]. Easy/Medium. Caveat: tight latency in hold loop → background thread, surface on pause, never block injection. → adr-v2-016.

  5. Continuous voice-biometric gate + anti-deepfake — inject only when live speaker matches the enrolled voiceprint; reject synthetic/replay. Anchor: ASVspoof 5 (arXiv 2601.03944), AASIST/SSL (arXiv 2502.03559). Extra [voiceguard]. Medium. Caveat: false-reject risk → off by default + override. → adr-v2-018.

  6. Emotion / tone-aware formatting — affect→formatting (auto !, emphasis) beyond pause→sentence. Anchor: emotion2vec / ParaS2S (arXiv 2511.08723). Extra [affect]. Easy/Medium. Caveat: SER is speaker/culture-variable → conservative default (emphasis + ! only). → adr-v2-017.

  7. Neural-codec ultra-low-latency streaming — streaming-native STT ~80ms. Anchor: Kyutai STT + Mimi codec (Jun 2025), MLX on-device. Extra [streaming-codec]. Medium. Caveat: English/French-centric → keep faster-whisper default multilingual. → adr-v2-022.

  8. Voice-grounded RAG over personal notes/docs — ask by voice, retrieve+cite from local docs. Anchor: EmbeddingGemma (308M, Sep 2025) + sqlite-vec + Gemma 3 270M. Extra [rag]. Medium. Caveat: hallucination → require inline citations, extractive fallback. → adr-v2-020.

  9. Silent-speech / subvocal (sEMG) — dictate by mouthing silently. Anchor: microneedle SSI 8.5% WER (S2666053925001249), emg2speech (arXiv 2510.23969); extends the existing EMGBackend. Extra [silentspeech]. Hard. Caveat: hardware + accuracy bottleneck → experimental, opt-in only. → adr-v2-023.

  10. Pure-vision screen commanding (VLM) — "click the blue Export button" where AT-SPI is empty. Anchor: Microsoft OmniParser V2, ShowUI, Florence-2. Extra [screenvision]. Hard. Frames stay in-RAM only (see adr-011). Caveat: on-device VLM latency → scope to icon+text grounding, AT-SPI primary. → adr-v2-024.

  11. Atypical-speech personalization via corpus LoRA — adapt the acoustic model to dysarthric/accented speech. Anchor: Universal Personalizer (arXiv 2509.15516), dysarthric LoRA 13.9% WER Euphonia (arXiv 2505.12991); extends Dysfluency Mode + held-out gate. Extra [atypical]. Hard. Caveat: on-device fine-tune is compute-heavy → idle/plugged-in only, validate on held-out data. → adr-v2-021.

Boundaries vs existing features

  • 8 RAG ≠ Spoken Recall (generative+cited over arbitrary docs, not search over past dictations).

  • 2 Translate ≠ Code-Switch (cross-lingual output vs keep-native spans).

  • 10 VLM ≠ AT-SPI Pilot (vision fallback when the tree is empty).

  • Personal voice-clone read-back (Sesame CSM-1B) was an honorable mention (incremental over the existing Kokoro read-back).

Citations here have not been re-verified against research/verify_refs.py.