ADR-v2-103 — Pitch-Contour Vocal Gestures¶
Status: Accepted (2026-07-02) · Wave L Context links: [[adr-v2-095-vocal-joystick]] (analog vs symbolic), gesture (keyboard chords), commands (word-based), [[adr-011]]
Context¶
Wave L research (#9) — assign short pitch contours (rising = confirm, falling = cancel, rise-fall = undo, flat-hum = repeat) as a word-free, language-independent command vocabulary: hum a shape to fire a command in a noisy room or without articulating words. Distinct from the Vocal Joystick (analog position control) and from every word-based command: this is a discrete symbolic channel built purely from F0 shape, robust in noise, usable by people who can phonate melody but not clear speech. Anchor: the non-speech vocal-interaction line (Bilmes/UW Vocal Joystick) + whistled-language phonetics (Meyer, Whistled Languages, Springer 2015) — pitch contour alone carries a full symbolic channel.
Decision¶
The default map stated above (flat-hum = repeat) is superseded by owner decision 1 of the 2026-10-21 amendment below: a flat hum is unmapped by default.
Add an opt-in Pitch-Contour Vocal Gestures: [contour] enabled=false. Pure cores in contour/gesture.py: normalize_contour(f0_series) → a z-normalized voiced contour, classify_contour(f0_norm) → rise | fall | rise_fall | flat, and gesture_to_command(gesture, profile) → a command name (a default map, user-overridable). The F0 tracker is the lazy extra; the sequence classifier is pure. OFF by default.
Consequences¶
- A discrete, language-independent, noise-robust command channel for dysarthria/anarthria users who retain pitch control.
- Pure classifier over an F0 array → fully testable and deterministic.
- Distinct from Vocal Joystick (symbolic vs analog) and word commands.
- Privacy (ADR-011): local F0 only.
- Caveat: a small fixed gesture set (four contours); off by default.
Amendment (2026-10-21) — Accepted (owner decisions, 2026-10-21)¶
Status of this amendment: accepted 2026-10-21. It was drafted as a proposal while building toward wiring (#164, WIRE-CONTOUR-001); the owner then decided open points (a) and (b) below (see Owner decisions), and the feature is now wired. [contour] stays off by default and is marked experimental; point © stands as a known limitation.
What was missing. The decision classifies an F0 contour, but nothing on main produced one. A shared numpy YIN tracker now does (src/yazses/audio/pitch.py, also used by ADR-v2-101), and src/yazses/contour/burst.py turns a finished burst into a smoothed voiced contour for the classifier above. No optional extra is involved.
The new decision: when is a burst a gesture at all? Every voiced word has a contour — a spoken "no" falls like the cancel hum — so classifying every burst would turn dictation into commands. Proposed gate (is_nonword_voiced_burst), both halves required:
- the transcript is empty or only hum spellings (
hmm,mm,mhm,uh,um,ah,er…). Any other word, in any script, keeps the burst as dictation — including spellings that are also words or letters (hum,err,em,m,u,n); - at least 80% of the burst's active frames are voiced, and at least 400 ms of them are.
Open points for the owner: (a) the default map sends a flat hum to repeat, so a lone thinking "hmm" would fire it unless 400 ms (a guess, unmeasured) separates deliberate hums from reflexive ones — consider leaving flat unmapped by default; (b) the #598 speech-presence check runs before transcription and may discard a hum, so the wiring must choose whether the contour check runs before it; © the thresholds were checked on synthetic tones only, not on recorded hums.
Owner decisions (2026-10-21)¶
- A flat hum is unmapped by default (open point a). The default map is now rise → confirm, fall → cancel, rise-fall → undo;
flatmaps to nothing, so a reflexive thinking "hmm" does nothing. A user profile passed togesture_to_commandmay still map it. - With
[contour]enabled, the hum check runs before the #598 no-speech discard (open point b), so a hum is not thrown away first. With[contour]disabled the #598 gate is byte-for-byte unchanged.
How it is wired¶
Daemon._contour_gesture calls contour/wiring.py::run_hum_gesture at two points in _on_hold_end, and returns at once without importing anything while [contour] is off:
- Before the #598 discard. When the speech detector has found no speech in a burst, the burst gets the hum check before it is discarded. No transcript exists yet, so the detector's positive "no speech" finding stands in for the empty-transcript half of the gate; the mostly-voiced half applies in full. A burst that is not a recognised hum is discarded as before. This is the stricter reading of "before": a burst the detector takes for speech is never classified here, because without a transcript a spoken, mostly-voiced "no" would pass the gate as a falling hum.
- After decoding, before the empty-transcript discard. A burst that reached the decoder is a gesture only if the transcript is empty or hum spellings only and the audio is mostly voiced.
A recognised gesture runs an existing path and types nothing: confirm / cancel answer a held confirmation through the same holder as the spoken "confirm" and the face switch (a no-op when nothing is held); undo deletes the last burst YazSes typed, as "scratch that" does (refused with no editable target, or with [revise] off). Anything else falls through unchanged. Tested on synthetic tones only (tests/test_contour_daemon_wiring.py).