YazSes v0.3.0 — Build Prompt for Claude Code¶

Repository and Architecture¶

You are implementing YazSes v0.3.0. The repository is at /home/mohsen/scratch/yazses. The project is a cross-platform, offline hold-to-talk voice dictation daemon (Linux/macOS/Windows). It uses Python 3.11+, managed with uv. The test command is uv run pytest tests/ -v.

Existing pipeline (do not break):

Hotkey hook → AudioRecorder → FasterWhisperEngine → postprocess/cleaner.py → inject/auto.py

The daemon orchestrator is src/yazses/core/daemon.py. Platform abstractions (hotkey, injector, IPC server/client, lifecycle, tray) are in src/yazses/platform/. All platform backends implement Protocol interfaces declared in src/yazses/platform/base.py. Config is a dataclass tree in src/yazses/config.py, loaded from TOML at ~/.config/yazses/config.toml. IPC uses JSON-RPC 2.0 over newline-delimited JSON on a Unix socket (Linux/macOS) or named pipe (Windows), implemented in src/yazses/ipc/.

Files to Create¶

Create all of the following new files. Do not modify existing files unless explicitly instructed.

cap-001 — SSH/Remote Voice Forwarding¶

Implementation notes for cap-001: - The SSH command is: ssh -o ExitOnForwardFailure=yes -R 9875:127.0.0.1:9875 [-i key_file] [-p port] host yazses-agent --listen 9875 - forwarder.py must catch FileNotFoundError (ssh not installed) and raise a descriptive YazSesError. - local_proxy.py uses asyncio streams to accept connections on 127.0.0.1:9875. - agent.py is the entry point for the yazses-agent CLI command (add to pyproject.toml [project.scripts] as yazses-agent = "yazses.remote.agent:main"). - Session state: add REMOTE_SETUP and REMOTE_ACTIVE to TrayState enum in platform/base.py. - Add IPC methods remote_start, remote_stop, remote_status to ipc/server.py. - Add CLI subcommand yazses remote --host to cli.py that calls remote_start over IPC.

cap-002 — Streaming Transcription with Real-Time Display and Correction¶

Implementation notes for cap-002: - PartialHypothesis is a dataclass with fields text: str, is_stable: bool, char_count: int. - The stable prefix comparison is: common = "" then iterate chars of both strings in parallel until mismatch. The stable delta is common[len(last_emitted):]. - StreamingEngine runs in a threading.Thread started by daemon._build_pipeline() when config.streaming.enabled = True. - The audio queue is an asyncio.Queue[np.ndarray]; audio chunks are pushed by AudioRecorder._callback when streaming is active. - Add inject_key_sequence(keys: list[str]) to InjectorBackend Protocol in platform/base.py. Implement in each platform injector: Linux xdotool (xdotool key shift+Left), macOS CGEventPost with shift modifier, Windows SendInput with VK_LEFT + SHIFT. - Add config dataclass StreamingConfig to config.py: enabled: bool = True, partial_interval_ms: int = 300. - Add streaming_enable IPC method to ipc/server.py.

cap-003 — Code Command Grammar¶

Implementation notes for cap-003: - CommandIntent is a dataclass: intent: IntentType, action: str, args: dict[str, str], raw_text: str. - IntentType is an Enum: DICTATE, NAVIGATE, EDIT, REFACTOR, TERMINAL. - GrammarSpec is a list of (re.Pattern, IntentType, str) tuples (pattern, intent, action_name). - Implement all 12 commands from the PRD FR-003.3 table as compiled regex entries. - Number word normalisation: implement a _normalise_numwords(text: str) -> str helper that replaces "one"→"1", "two"→"2", "three"→"3" (through "ten"→"10") before pattern matching. - dispatch.py action table maps action names to key sequences: "undo" → ["ctrl+z"], "save" → ["ctrl+s"], "go_to_line" → ["ctrl+g", "{n}", "Return"], etc. - Add config dataclass CommandsConfig to config.py: enabled: bool = True, profile: str = "auto", custom: list[dict] = [].

cap-004 — Offline Disfluency Filter¶

Implementation notes for cap-004: - FilterResult is a dataclass: text: str, chars_removed: int. - Rule A: compile re.compile(r'\b(' + '|'.join(re.escape(w) for w in config.filler_words) + r')\b[,]?', re.I). Substitute with "". Then normalise multiple spaces. - Rule B: tokenise by whitespace. Walk with a sliding 2-gram window; if tokens[i:i+2] == tokens[i+2:i+4], remove the second occurrence. Repeat until stable. - Rule C: scan for any trigger phrase from config.self_correction_triggers. If found, remove from the last sentence boundary (., !, ?) before the trigger, through to the end of the trigger phrase. - Proper noun and code identifier guard: apply rules only to tokens that are all-lowercase or match \b(um|uh|er|...)\b. Do NOT strip tokens that contain uppercase letters (proper nouns) or contain _, /, . (code identifiers). - Add DisfluencyConfig and FiltersConfig to config.py with all fields from the architecture doc (§5.4). - Wire disfluency.filter_transcript() into core/daemon.py in _process_final_transcript(), after postprocess/cleaner.py and before commands/grammar.py.

cap-005 — Accessibility Profile¶

Implementation notes for cap-005: - enroll.py: for each of 20 utterances, record 3 s using AudioRecorder, compute noise_floor = np.abs(audio[:int(0.5 * 16000)]).mean() and speech_rms = np.abs(audio[int(0.5 * 16000):]).mean(). After all 20: vad_threshold = np.percentile(noise_floors, 95) * 3.0, min_silence_ms = max(500, int(np.percentile(pause_durations, 95))). Write to config with tomllib/tomli_w. - platform/linux/hotkey.py: add evdev_device: str = "" to config lookup. If config.hotkey.source == "evdev" and config.hotkey.evdev_device != "", open that device path instead of the default keyboard device. - Add CLI command yazses enroll to cli.py that calls accessibility/enroll.py:run_wizard(). - Extend yazses doctor to accept --accessibility flag: print current values of vad_threshold, min_silence_ms, pre_speech_padding_ms, hotkey.evdev_device, stt.model. - Add AccessibilityConfig and RemoteConfig to config.py (see architecture doc §5.4 for all fields and defaults).

Config Schema Changes¶

Modify src/yazses/config.py to add the following dataclasses (all fields have safe defaults so existing configs remain valid):

@dataclass
class StreamingConfig:
    enabled: bool = True
    partial_interval_ms: int = 300
    partial_marker: str = ""

@dataclass
class DisfluencyConfig:
    enabled: bool = True
    filler_words: list[str] = field(default_factory=lambda: [
        "um", "uh", "er", "ah", "hmm", "like", "you know",
        "i mean", "basically", "right", "okay so"
    ])
    self_correction_triggers: list[str] = field(default_factory=lambda: [
        "no wait", "delete that", "scratch that", "never mind",
        "forget that", "strike that"
    ])
    llm_enabled: bool = False
    llm_endpoint: str = "http://localhost:11434"

@dataclass
class AccessibilityConfig:
    min_silence_ms: int = 500
    pre_speech_padding_ms: int = 200
    vad_source: str = "default"
    vad_threshold: float = 0.01

@dataclass
class CommandsConfig:
    enabled: bool = True
    profile: str = "auto"
    custom: list[dict] = field(default_factory=list)

@dataclass
class RemoteConfig:
    default_host: str = ""
    ssh_port: int = 22
    agent_port: int = 9875
    key_file: str = ""

Add these as fields to the top-level Config dataclass: streaming, filters, accessibility, commands, remote.

Architecture Decisions (Already Made — Do Not Relitigate)¶

All design decisions are locked. Implement exactly as specified:

• ADR-001: Only transcribed text crosses the SSH tunnel. No audio bytes leave the local machine. remote/forwarder.py and remote/local_proxy.py have zero imports of sounddevice or audio.*.
• ADR-002: Use LocalAgreement 2-iteration policy for streaming partial hypotheses. Do not emit raw Whisper output. The stable prefix is the longest common prefix between two consecutive decode passes.
• ADR-003: Command classification is a compiled-regex grammar classifier on text output from Whisper. No second ASR model, no fine-tuning, no LLM classifier.
• ADR-004: Correction-on-commit uses Shift+Left × _chars_injected to select partial text, then injects final text to replace. No clipboard operations. No floating overlay window.
• ADR-005: Accessibility in v0.3.0 is config parameters + enrollment wizard only. No LoRA fine-tuning. No new ML dependencies.

Tests to Write¶

For each new module, create a corresponding test file in tests/. Write tests that do not require real hardware (mock sounddevice, evdev, xdotool, subprocess):

Create the test fixture files: - tests/fixtures/disfluency/corpus.json — 100 entries as described in the eval plan. - tests/fixtures/commands/command_phrases.json — 50 entries, one per command from FR-003.3. - tests/fixtures/commands/dictation_corpus.txt — 500 words of prose. - tests/fixtures/streaming/ — at minimum 3 WAV files (synthetic, 16 kHz, float32, 3 s). - tests/fixtures/accessibility/calibration_utterances/ — 20 WAV files (synthetic, 16 kHz, float32, 3 s). - tests/fixtures/accessibility/rms_stats.json — expected RMS statistics for the 20 fixtures.

Pass/Fail Criteria¶

All of the following must be true before reporting complete:

Version Bump¶

After all tests pass, update the version in two places:

1. pyproject.toml: change version = "0.2.4" to version = "0.3.0".
2. src/yazses/__init__.py: set __version__ = "0.3.0".

Verify: uv run yazses --version outputs YazSes 0.3.0.

Run uv run pytest tests/ -v and confirm all tests pass before reporting complete.