Spec: Face-Gesture Switch — Calibrated Deliberate Expression as an Accessibility Input¶

Goal¶

Turn selected deliberate facial expressions into robust abstract switch events without letting the detector execute actions directly.

What already exists. yazses.facegesture.detector.GestureSwitch is a shipped, tested, pure hold detector with hysteresis and frame debounce over five named gestures, and yazses.facegesture.backend.FaceGestureBackend drives it from its own webcam. This spec does not re-specify that detector. It adds the two things ADR-v2-135 deliberately left out: a neutral per-user baseline instead of absolute thresholds, and a switch-intent boundary so the gesture is not hard-wired to hold-to-talk. Everything below that describes a detector is a change to the shipped one, not a new module.

Non-goals¶

• no identity recognition;
• no emotion inference;
• no medical assessment;
• no raw-image storage;
• no default blink-to-click;
• no recommended threshold before field measurement.

Input¶

A timestamped FaceSignal from shared perception containing selected blendshape scores and quality.

Detector tests consume numeric traces directly.

Calibration¶

Before active detection, collect a short neutral baseline.

Calibration output may include: - neutral mean/median per required blendshape; - robust variation estimate; - derived threshold offsets.

Do not store raw frames. Persist only derived calibration values when persistence is useful.

Calibration must be repeatable/resettable.

Gestures¶

Initial supported candidates: - mouth open; - eyebrow raise; - optional smile; - blink advanced/opt-in only.

Each configured gesture declares the blendshape combination it uses.

Avoid depending on one vendor-specific label outside the adapter; normalize names if needed.

State machine¶

Per gesture:

NEUTRAL
 -> CANDIDATE (enter threshold crossed)
 -> ACTIVE (held for minimum duration)
 -> FIRED / HELD
 -> WAIT_FOR_RELEASE (must cross exit threshold)
 -> REFRACTORY
 -> NEUTRAL

The implementation may collapse states but tests must establish equivalent semantics.

Hysteresis¶

enter_threshold and exit_threshold differ to prevent chattering.

Threshold interpretation is relative to the calibrated baseline wherever possible.

Time rules¶

Configurable: - minimum hold time; - refractory/debounce interval; - sample freshness timeout.

All timing uses monotonic source time/fake clock semantics.

As built (2026-10-20, #406). GestureSwitch counts the minimum hold (min_hold_frames) and the refractory interval (refractory_frames) in samples. It also takes the same two rules in seconds, min_hold_s and refractory_s, measured on the caller's now_s sample timestamps. Both default to 0 (off), and each adds to its frame rule instead of replacing it. The freshness timeout is gesture_reading(max_age_s=...) and the stuck-open bound is max_hold_s. The timed rules fail closed: - an untimed sample breaks a hold run and keeps a refractory window open; - a gap over max_gap_s (provisional 0.25 s) restarts the run; - a repeated sample adds no time; - a backwards clock step restarts the count.

The timed rules are why frame counts alone do not meet "minimum hold time". A frame count's length is whatever the source's rate makes it, and the shared camera's rate ([perception] fps) is not the face switch's own fps. That gap was a live defect for blink (#406). The panel's eyes-closed floor of 12 frames ("0.8 s at 15 fps") lasted 0.4 s on a 30 fps shared camera, inside a long blink. So every live eyes-closed detector now carries min_hold_s = EYES_CLOSED_MIN_HOLD_S (0.8 s, provisional) through detector.min_hold_s_for, on top of its frame count. That covers the face-switch backend on both camera paths and the chord sensors (gesture/sensors.py), and both pass sample time. The 0.8 s is an internal safety floor, not a config key. Other gestures' timed rules are not yet exposed as [facegesture] config keys. That is a public-interface change for a follow-up. No duration may be called recommended until it has been measured in the field.

Event contract¶

Detector emits a normalized event such as: - kind; - phase/fired; - timestamp; - confidence.

It contains no PointerSink/action reference.

Mapping layer¶

Separate adapter maps event -> abstract action/token: - hold-to-talk; - select/click; - confirm; - cancel; - pause/resume; - Gesture Chord token.

Unknown/unmapped events do nothing.

A stop or a re-arm of the hands-free gate is a boundary for both consumers of the face channel (#417, ADR-v2-148). The face switch (SwitchMapper.frame) and Gesture Chords (ChordEngine.frame, #408) each read the gate's one-shot clear_pending latch before a frame is counted. Each then drops every half-accumulated gesture. For chords that includes the head detector's half-made nod, shake or tilt. A gesture begun before a Resume must therefore be made again, in full, after it. An open hold stays held and must be relaxed before it can press again.

Destructive desktop operations still use the existing consequence/confirmation policy.

As built (EYE-FACE-002, #407). facegesture/switches.py (SwitchIntent, parse_bindings, SwitchMapper) is the mapping layer; facegesture/actions.py holds the pure decisions behind select / confirm / cancel; the daemon supplies the handlers. The vocabulary is hold_to_talk, hold_to_command, select, confirm, cancel, pause_pointer (and none to unmap). Holds enter the daemon through the hotkey's own entry points, so cmdsafety still holds a destructive spoken command; confirm / cancel only answer a confirmation that already exists. pause_pointer does the most it can in each configuration, matching handsfree pause and a chord's pause_pointer: with [handsfree_safety] on it pauses the global gate; with it off it pauses a running Head-Pointer only; with neither it stays unwired and the daemon names the setting to turn on. A Gesture Chord token is not a face-switch action: chords consume face tokens through their own runtime (EYE-GESTURE-001, gesture/sensors.py).

Talking interference¶

Mouth-open is explicitly tested against traces resembling normal speaking motion.

Until real false-activation evidence is available: - mouth-open mapping remains experimental; - no claim that it is safe while dictating; - users can choose eyebrow or other deliberate gesture if measured better.

Blink policy¶

Natural blink frequency makes blink a poor default commit signal.

Blink: - never ships as default click; - may exist as an advanced mapping; - needs its own false-activation evidence.

Head-Pointer coupling¶

The composition layer can assert a clutch while an ACTIVE commit gesture is underway. This prevents the expression/head movement from shifting the target simultaneously.

Config¶

A dedicated optional section may include: - enabled; - primary/secondary gesture; - action mapping; - per-gesture hold/refractory/threshold adjustments; - calibration values/path/version; - confidence minimum.

Config validation rejects unknown gestures/actions and unsafe malformed thresholds.

Acceptance criteria¶

• Neutral calibration emits no event.
• Brief threshold crossing shorter than minimum hold emits no event.
• Sustained deliberate trace emits one event.
• Detector must release before re-arm.
• Refractory prevents bounce.
• Low-confidence/stale samples emit no event.
• Ordinary blink trace does not fire a non-blink gesture.
• Talking-like mouth trace is included in regression fixtures.
• Detector never imports daemon/pointer modules.
• Unmapped event causes no action.

Measurement¶

Primary field metric: false activations per hour by configured gesture.

Also collect: - deliberate attempts; - detected attempts; - misses; - latency; - environment notes.

Synthetic traces are regression tools, not evidence for recommended defaults.