Work with YazSes: students, researchers, industry¶
YazSes is a production dictation tool and an open research platform for voice-first human-computer interaction. Everything runs on-device, every subsystem sits behind a pluggable seam, and the project is small enough that one person can still understand the whole pipeline — which makes it unusually good material for coursework, theses, papers, and pilots.
This page maps what the project offers to each audience. Questions and proposals are welcome in GitHub Discussions — the Research Corner threads are the standing venue for scientific discussion.
flowchart TD
Q{What do you want<br/>to walk away with?} --> A[A graded project<br/>or a thesis]
Q --> B[A publishable<br/>measurement]
Q --> C[A deployment<br/>in my organisation]
A --> A1["Pick from the scoped projects below<br/>— each has a defined evaluation"]
B --> B1["Take an open research question<br/>from the survey pages"]
C --> C1["Read the privacy statement and<br/>Meeting Mode, then open a Discussion"]
A1 --> D[Open a Discussion to claim it]
B1 --> D
C1 --> D
D --> E["Fast review · co-authorship where earned ·<br/>your name in the release notes"] Why YazSes works as a research platform¶
The architecture separates every scientifically interesting component behind a small interface, so an experiment replaces one box without touching the rest:
| Seam | Interface | Swap in… |
|---|---|---|
| Speech-to-text engine | SttEngine (stt/base.py) | your model — Whisper variants and NVIDIA Parakeet TDT already ship |
| Voice-activity detection | [meeting] vad_backend, audio/vad_calibrated.py | energy gates, Silero, your VAD |
| Speaker diarization | recimport/diarizer.py | sherpa-onnx by default; the pyannote backend shipped in #71 |
| Speaker embeddings | voiceprint/ | ECAPA by default; the lighter Resemblyzer backend shipped in #70 |
| Activation source | HotkeyBackend protocol | keyboard, EMG squeeze (YESP serial), your sensor |
| Gaze targeting | gaze/ backends | MediaPipe iris today; your estimator |
Two properties matter for studies: everything is offline (no cloud confound, works in a lab without agreements with a third-party processor), and the opt-in learning corpus is encrypted on-device, which is the right starting point for an ethics/IRB conversation rather than an obstacle.
For students: project-sized problems¶
Each of these is a real, wanted contribution with a defined evaluation and a maintainer who reviews quickly. They are sized between a course project and a bachelor/master thesis:
| Project | Where it stands | Flavour |
|---|---|---|
| Measure Glance-Type's real gaze accuracy on ordinary webcams | open — #104 · background | HCI / multimodal |
| Evaluate Dysfluency-Friendly mode with people who stutter | open — #255 · use case | accessibility / NLP |
The seams below shipped between August and September 2026. They are no longer open work, but the issue behind each one records the design argument and the measurements that settled it, which is the useful starting point if you want to extend or re-evaluate one:
| Shipped seam | Design record | Flavour |
|---|---|---|
| STT benchmark harness — WER/RTF methodology + community results | #72 | empirical / reproducibility |
| Vocabulary correction after decoding (Parakeet ignores prompts) | #73 | speech ML, open design |
| Streaming STT: Moonshine engine + latency study | #74 | systems / real-time |
| Noise suppression in the dictation hot path (DeepFilterNet) | #69 | audio DSP |
| Speaker-embedding reliability on sub-second windows | #70 | speaker modelling |
| Diarization accuracy vs speed trade-off (pyannote vs sherpa) | #71 | evaluation |
| Persian and RTL portable contract vectors | #512 | evaluation / i18n |
Supervisors: if you want to run one of these as a course assignment or thesis, open a Discussion — scoping a variant for your semester is exactly the conversation we want to have.
For researchers: evaluation, citation, collaboration¶
- Cite the project with the repository's
CITATION.cff(GitHub's Cite this repository button), which also references the companion preprint (arXiv:2607.28878) — the BibTeX entry is on the research index. - Reproducibility: pinned Python dependencies, CPU-int8 inference, offline models — a described configuration re-runs bit-for-bit on commodity hardware. The benchmark harness (#72) shipped but is no longer part of this repository; its published results, with their method, are in Benchmarks.
- Study directions already grounded in the codebase: multimodal input (gaze + voice + EMG), personal VAD and the cocktail-party problem on short windows, code-switching dictation (Polyglot), privacy-preserving on-device personalization (encrypted corpus + local fine-tuning), and dictation UX for people with RSI or dysfluent speech.
- The research section surveys the literature behind the post-keyboard input roadmap (eye control, voice control, muscle & brain interfaces).
For industry: private dictation you can actually deploy¶
- Nothing leaves the machine — see the privacy statement. No per-seat cloud subscription, no audio retention, no data-processing agreement needed for the dictation path.
- Meeting transcription on-premises — Meeting Mode produces speaker-labelled transcripts and minutes without audio ever reaching a third party.
- Scriptable deployment — plain TOML config, a CLI for everything, systemd/launchd lifecycle (Windows autostarts per-user; there is no Windows Service backend yet), and a container image for the headless transcription pipeline on GHCR (#76 — see Docker).
- Pilots and integration questions: open a Discussion or contact the maintainer (see the repository profile). Honest expectations: this is an open-source project, not a vendor with an SLA — what you get is source access, fast issue turnaround, and a say in the roadmap.
For everyone: the community rhythm¶
- Research Corner (Discussions) — a recurring scientific thread: one paper, question, or experiment per thread, tied where possible to something measurable in YazSes. Bring results, not just links.
- Contributor lanes — the pinned issue groups every open task by interest: GUI, packaging, speech/ML, voice-control features, docs and no-code testing.
- Benchmarks — the harness has landed; add a row for your machine.