Skip to content

ADR-002: Dual-Stack STT Routing

Status: Accepted
Date: 2026-05-18
Deciders: Mohsen Seyedkazemi Ardebili

Context

Short voice commands (under 4 seconds) and long-form dictation have different latency and accuracy requirements that no single 2026 STT model satisfies simultaneously. Whisper-class encoder-decoder models pay the cost of a 30-second windowed encoder on every utterance, producing 300–500 ms first-token latency even for one-second commands on consumer CPU. Edge-optimised streaming models (Moonshine, Parakeet-TDT) achieve sub-100 ms first-token latency by using variable-length input, but Moonshine is English-only at release and lacks Whisper's initial_prompt mechanism needed for editor-LSP context injection (see ADR-006).

Decision

A dual-stack STT architecture is adopted for v1.0. STTRouter selects the path based on Silero VAD duration estimate, with a user-configurable threshold defaulting to 4 seconds:

  • Streaming path (≤ 4 s): Moonshine v2 small-streaming (moonshine-voice ≥ 0.0.59, MIT). Measured P50 latency: 9 ms on i7-1370P. Fallback to Moonshine v1 base ONNX (77 ms P50) on architectures where the native library is unavailable.
  • Long-form path (> 4 s): Whisper-large-v3-turbo via whisper.cpp (all platforms) or MLX-Whisper (Apple Silicon fast path). Accepts initial_prompt from the EditorBridge.

v1.1 will add Parakeet-TDT-0.6B-v3 as a multilingual streaming path for non-English users, and Canary-1B-v2 as an alternative long-form path.

Consequences

Positive: - P50 first-token latency for short commands drops from ~431 ms (faster-whisper tiny.en int8) to 9 ms on the streaming path. - Long-form accuracy is retained at Whisper-large-v3-turbo level. - The initial_prompt LSP-context injection (cap-002) is preserved on the long-form path. - Apple Silicon users get an MLX-Whisper fast path automatically. - The STTBackend Protocol accommodates a third engine in v1.1 without any retroactive changes.

Negative / trade-offs: - Two model families at install time: Moonshine v2 small-streaming (~235 MB) and Whisper-large-v3-turbo (~800 MB int8). - Routing on partial-duration estimates adds logic complexity at the threshold boundary. - v1.0 ships English-only on the streaming path; multilingual users use the Whisper long-form path for all utterances until v1.1.

Implementation

The STTRouter is implemented in yazses-stt/src/router.rs. Backend adapters live in yazses-stt-onnx/ (Moonshine ONNX) and yazses-stt-whispercpp/ (whisper.cpp FFI). Silero VAD duration estimation runs in yazses-audio/src/vad.rs and feeds the routing decision before recording completes.