Research notes¶
Studies, surveys and the literature watch behind the design decisions.
This is the engineering tier
These pages are written for contributors, reviewers and researchers rather than for people using YazSes. For the user documentation, start at Home.
69 documents.
- Idea mining: silent speech & neural interfaces (2026-08-08)
- The YazSes HCI reference corpus — 105 verified references
- The YazSes HCI research agenda — ten directions
- Adjacent open-source projects, and what is worth taking from them
- Modular distribution: what a YazSes install actually costs, and how other projects solve it
- The problem space — what a future YazSes would be for
- Futuristic Voice-First HCI Capabilities for YazSes — Research Scope
- State of the Art: Futuristic Voice-First HCI for YazSes — 2025
- Gap Analysis: Futuristic Voice-First HCI for YazSes — 2025
- YazSes v0.4 — Capability Prioritization Matrix
- YazSes v0.4 PRD — Offline LLM Routing, Code-Aware Dictation, EMG Backend
- YazSes v0.4 Architecture — Offline LLM Routing, LSP Context, EMG Backend
- Use llama-cpp-python (in-process GGUF) for Tier 2 SLM inference
- Use pygls JSON-RPC client for LSP context extraction
- EMG backend uses USB CDC serial with hardware-agnostic message protocol
- Offline LLM Intent Routing Layer
- Code-Aware Voice Dictation via LSP Context Injection
- EMG Silent Speech Backend
- Towards Spatial Computing: Recent Advances in Multimodal Natural Interaction for XR Headsets
- VoiceBench: Benchmarking LLM-Based Voice Assistants
- Adaptive Edge-Cloud Inference for Speech-to-Action Systems Using ASR and LLMs
- Natural Multimodal Fusion-Based HRI: Voice and Deictic Posture via LLM
- Wireless Silent Speech Interface Using Multi-Channel Textile EMG Sensors Integrated into Headphones
- Typing Reinvented: Towards Hands-Free Input via sEMG
- Sentence-Level Silent Speech Recognition Using Wearable EMG/EEG Sensor Fusion
- Affective Mirroring in Video Game NPCs: A Pilot Study Evaluating Player Engagement
- LLM-Driven NPCs: Cross-Platform Dialogue System for Games and Social Platforms
- A Systematic Review of Fusion Methods for the User-Centered Design of Multimodal Interfaces
- Biosignal-Enabled Spoken Communication — Interspeech 2025 Special Session
- AI Applications for Interpreting Communicative Movements by Individuals with Visual and/or Motor Disabilities
- YazSes Innovation Roadmap — Research Scope
- YazSes Innovation — State-of-the-Art Matrix
- YazSes Innovation — Gap Analysis
- YazSes Innovation — Capability Prioritization Matrix
- YazSes v0.3.0 — Product Requirements Document
- YazSes v0.3.0 — Architecture Document
- YazSes v0.3.0 — Evaluation Plan
- YazSes v0.3.0 — Production Readiness Checklist
- YazSes v0.3.0 — Build Prompt for Claude Code
- ADR-001: Text-Only SSH Forwarding (No Audio-Over-SSH)
- ADR-002: LocalAgreement Policy for Streaming Partial Hypotheses
- ADR-003: Grammar-Rule Intent Classifier (No Dedicated Coding ASR Model)
- ADR-004: Correction-on-Commit via Selection-Replace
- ADR-005: Accessibility via Config Parameters First, LoRA Fine-Tune Second
- YazSes Remote — Voice Dictation Over SSH Sessions
- Streaming Transcription with Real-Time Display and Rollback Correction
- Code Command Grammar — Voice-Driven Code Editing Actions
- Offline Disfluency Filter — Clean Text Before Injection
- Accessibility Profile — Personalised Whisper Adaptation for Atypical Speech
- XR Voice API — WebSocket JSON-RPC Event Server for AR/VR/Metaverse
- Gaming Profile — Spatial Voice Commands for 3D Games and Simulation
- LLM Intent Layer — Adaptive Local LLM for Command Disambiguation
- faster-whisper: Faster Whisper transcription with CTranslate2
- whisper_streaming — Online Whisper STT with LocalAgreement Policy
- Turning Whisper into Real-Time Transcription System (arXiv 2307.14743)
- WhisperPipe — Low-Latency Streaming ASR with Consensus Engine (arXiv 2604.25611)
- WhisperX — Forced Alignment and Speaker Diarization for Whisper
- Serenade — Voice-Controlled IDE Plugin with Speech-to-Code Model
- Talon Voice — Gold Standard Hands-Free Coding with Python Scripting API
- Voiceitt — Personalised ASR for Atypical Speech
- ASR in 2025-2026: A Deep Dive — Practitioner Benchmark and Stack Comparison
- When to Call the LLM: Adaptive Speech Interfaces for NL Input in XR (ACM VRST 2025)
- Human–3D Object Interaction Through Voice Commands in Immersive VR (MDPI 2025)
- Back to Basics: Revisiting ASR in the Age of Voice Agents (arXiv 2603.25727)
- Superwhisper — Local Whisper Dictation with LLM Post-Processing and Per-App Profiles
- Towards Spatial Computing: Multimodal Natural Interaction for XR Headsets (Frontiers of CS, Springer 2025)
- A2-LLM: End-to-End Conversational Audio Avatar LLM (arXiv 2602.04913)
- ASR for Non-Native English: Accuracy and Disfluency Handling (arXiv 2503.06924)
- Research digest — 2026-08-15