Offline Chinese voice typing on a PC, with YazSes¶

What this page is: a how-to for people who already use YazSes for English and want to point it at Mandarin anyway, plus one Chinese-specific problem we did fix (the Simplified/Traditional trap) and the numbers we measured while fixing it.

It is deliberately specific about what works and what does not, because Chinese support in offline speech recognition is usually oversold. There is also a Simplified Chinese version of this page.

It is not on by default — you must switch models¶

YazSes ships with base.en, an English-only Whisper checkpoint. English-only checkpoints have no language tokens at all, so they cannot decode Mandarin: fed Chinese speech they produce fluent-looking English nonsense rather than an error, which is the worst possible failure mode. YazSes warns about this at start-up, but you still have to change the setting yourself.

The quickest route is one command that writes every key, installs what is missing and fetches the model:

yazses language set zh-CN      # or zh-TW; add --dry-run to preview the plan

To do it by hand instead, in ~/.config/yazses/config.toml:

[stt]
model = "small"                  # multilingual (no .en suffix): base / small / medium / large-v3
language = "zh"                  # Mandarin
chinese_script = "simplified"    # or "traditional" for Taiwan and Hong Kong

Then:

yazses features enable chinese-script   # installs the `chinese` extra
yazses restart

features enable chinese-script only pins the script. It does not change the speech language or the model, so it refuses, and writes nothing, until [stt] language = "zh" is set. Script conversion on English or auto-detected speech is a combination yazses doctor reports as a failure. The settings window refuses it the same way, and both point you at yazses language set zh-CN (or zh-TW).

Why chinese_script matters more than it sounds¶

Whisper decides per utterance whether to answer in Simplified or Traditional characters, and it is not consistent about it. Measured on 20 clean 16 kHz Mandarin utterances (the ASCEND test split, small model), 13 came back in Traditional characters — including ones where the recognition itself was completely correct. A mainland user dictating 简体中文 watches 繁體字 land in their editor.

That inconsistency costs far more accuracy than it appears to, because the recognition is usually right and only the script is wrong. Character error rate against Simplified references:

Same audio, same model, one config key. The setting matters most for the small models, which is exactly what CPU users run — small improves by 19 points, large-v3 by only one, because the larger model already tends to answer in Simplified. If you are dictating on a laptop CPU rather than a GPU, this setting is doing most of the work.

The measurement and its caveats are documented in postprocess/han_script.py.

Conversion is a reversible character mapping, so it cannot repair a mishearing — it only stops a correct transcription from arriving in the wrong script.

Chinese voice commands: experimental, safe core only¶

Mandarin voice commands are an experimental preview. No native Mandarin speaker has reviewed the phrases yet (#505), so every row in the table below is marked unreviewed. Expect phrasing that sounds unnatural to you, and please report it.

They are off on a default install, which uses the English grammar. They turn on only when you choose Mandarin: with [commands] language = "auto" (the default) the command grammar follows [stt] language, so a Mandarin language profile (yazses language set zh-CN or zh-TW) selects them. [commands] language = "zh" selects them explicitly.

[commands]
language = "auto"  # follows [stt] language; explicit "en" or "zh" overrides

To dictate Mandarin without Mandarin commands, pin language = "en" under [commands]. The English grammar never matches a Chinese sentence, so everything you say in Chinese is typed as text, and English commands such as "undo that" still work. enabled = false under [commands] stops the dictation key from running any command at all; only the dedicated command-mode key, if you bound one, still runs them.

A phrase is a command only when it is the whole utterance, with nothing before or after it apart from punctuation. "保存文件" saves; "请保存这个文件" or "保存文件之后再说" is typed as text. Each phrase below is one accepted wording; N stands for a number (see the limits further down) and name for one identifier or file name.

Each phrase sends the same command as its English equivalent: 撤销三次 and "undo 3 times" reach the editor as the same action with the same count. The tests check that this table and the grammar agree in both directions, and that the review column matches the review state recorded in the code.

This is intentionally not full parity with the English command grammar yet. Mandarin terminal execution (the English run ... family) and open-ended refactor/create commands are not registered in P1. An unmatched Chinese phrase remains dictation on the normal dictation key; in dedicated command mode it remains unmatched and is not executed. The Chinese Tier-1 path also does not fall through to the English SLM router — which is opt-in and off unless [commands] slm_model_path names a local model anyway — so ordinary Chinese prose cannot become a fuzzy English command.

Getting Chinese characters into the app¶

Recognising Mandarin is half the job; the other half is typing 汉字 into whatever window has focus, and that depends on your platform's injection backend. What each one does with a Han character, from the code — none of this has yet been confirmed on real hardware for Chinese text, which is tracked in #501:

On the ydotool path, a burst that mixes English and Chinese — YazSes 版本 v2.0 — is now checked before the first keystroke: if any character cannot be typed, nothing is typed and the whole burst is pasted once. Earlier versions typed the English part, failed on the first Han character, and then pasted the whole burst after it, so the English appeared twice.

Spacing follows Chinese typography: no space is inserted between two consecutive Chinese dictation bursts, or after full-width punctuation such as ， and 。. A space is kept where Chinese meets a Latin word or a number spoken as a separate burst (测试 YazSes).

Numbers in commands are bounded and exact. Counts ("撤销三次", "删除最后两行") run from 0 to 99, and line numbers ("转到第一百二十三行", "前往第 150 行") from 0 to 999. A form outside that set — 一千, the elided 一百一, 两十 — is typed as text, not guessed. Deletion only fires on the explicit "删除最后 … 词/行" form. Every Mandarin command phrase is provisional until a native Mandarin speaker has reviewed it; until then, treat the set as a preview.

Meetings and file transcripts¶

yazses transcribe and Meeting Mode keep Chinese text as written in every output format — Markdown, plain text, SRT, WebVTT and JSON. No space is inserted between characters, a speaker can have a Chinese display name, and renaming or merging speakers afterwards with yazses meeting relabel re-renders the transcript without adding spaces. Subtitle lines are measured in display columns (a Chinese character counts as two) and broken between characters. Meeting Mode's "this transcript looks too thin" check counts Chinese characters, not space-separated words.

If you turn on meeting notes ([meeting] notes, off by default), the local summariser reads the transcript in windows sized for its context. A Chinese character is budgeted as several tokens, a deliberately cautious upper bound, so a Chinese meeting is summarised in more, smaller windows than an English one of the same length. That costs time, not content. The token rate of a real summarisation model on Chinese text has not been measured.

All of this is checked with synthetic text in the test suite. It has not yet been checked with real recordings by a native speaker.

How accurate is it really?¶

Those figures come from ASCEND, which is spontaneous conversation by Hong Kong speakers with Mandarin-English code-switching — a deliberately hard case, and only 20 utterances. Read speech into a decent microphone in a quiet room does better; a noisy room or a strong regional accent does worse. Model size is the biggest lever available to you: on the same clips, large-v3 reached 11.3% where small reached 16.9%.

Do not take anyone's benchmark as a promise about your voice. Test it on your own audio before you rely on it:

yazses transcribe my-recording.m4a

Larger models are more accurate and slower to decode on CPU. small is a reasonable starting point; move to medium or large-v3 if your machine can afford the latency.

Chinese dictation in YazSes is best described as usable but still rough. Field reports are genuinely wanted — open an issue with what worked and what did not.

Why offline matters here specifically¶

Most Chinese voice input is either a website or a mobile keyboard: you speak into their text box and copy the result out, and it stops working the moment the network does. YazSes runs as a background program on your own computer, so it types straight into Word, a browser, a code editor, or a terminal — on a train, on a bad connection, or on a machine that never touches the internet at all.

That property is the point for anyone whose audio must not leave the building: clinical notes, legal drafting, unpublished research, interview recordings covered by an ethics approval, or work on an air-gapped machine. See private and confidential dictation and the privacy statement.

Related¶

• Multilingual dictation — the general non-English setup
• Hindi and Indian languages — the same pattern for another script
• Transcribe recordings offline
• 简体中文版本