Word error rate (WER) is the standard way to measure speech recognition accuracy. It counts the minimum number of word substitutions, deletions and insertions needed to turn a system's transcript into a trusted reference, then divides by the number of words in the reference. Lower is better: a WER of 5% means roughly one word in twenty is wrong.
Key takeaways
- WER = (substitutions + deletions + insertions) ÷ reference words. It is an edit distance computed on words, not characters.
- WER can exceed 100%, because insertions are unbounded.
- Normalisation decides the score as much as the model does. "€20" against "twenty euros" is an error unless both are normalised first.
- Every word counts the same. Dropping "the" costs as much as mishearing a drug name, which is why WER alone cannot tell you if a system is fit for a domain.
- A WER figure is only meaningful alongside its test set. Numbers from different datasets are not comparable.
Definition#
WER compares two strings of words: the reference, a human-verified transcript, and the hypothesis, what the speech recognition system produced. An alignment algorithm finds the cheapest way to edit the hypothesis into the reference, and each edit is one of three kinds:
- Substitution (S): a word was recognised as a different word.
- Deletion (D): a spoken word is missing from the transcript.
- Insertion (I): the transcript contains a word nobody said.
where N is the number of words in the reference. Because the denominator is the reference length and insertions have no upper limit, a system that outputs a long stream of hallucinated text can score above 100%.
A worked example#
Take a clinician's dictation and a hypothesis from a general-purpose model:
Three edits over six reference words gives 50%. A transcript that reads fluently and gets four of six words right is, by this measure, half wrong. And the one error that matters clinically, the drug name, is weighted exactly like the missing adverb.
Normalisation: the hidden variable#
Before scoring, both texts are usually normalised: lower-cased, punctuation stripped, and numbers, contractions and spelling variants mapped to one canonical form. Without this step, a system is penalised for writing Dr. Jansen when the reference says doctor Jansen, even though both are correct.
The choice of normaliser can move WER by several points on the same audio. Whisper's release popularised an English normaliser that expands or standardises numbers, British and American spellings, and filler words, and many leaderboards now use it or a close variant. Two vendors quoting WER with different normalisers are not quoting comparable numbers.
| Reference | Hypothesis | Raw | Normalised |
|---|---|---|---|
| It costs €20. | it costs twenty euros | 2 errors | 0 errors |
| We're organising it | we are organizing it | 3 errors | 0 errors |
| Call 112 now | call one one two now | 4 errors | 0 errors |
For languages where word boundaries are ambiguous, such as German or Dutch compounds, normalisation choices matter even more. See compound words and morphology.
Computing WER#
The alignment is a Levenshtein distance over word tokens, solved with dynamic programming in O(N·M) time. You rarely need to write it: the jiwer Python package is the common choice.
import jiwer
reference = "start metformin 500 milligrams twice daily"
hypothesis = "start met for 500 milligrams daily"
out = jiwer.process_words(reference, hypothesis)
print(out.wer) # 0.5
print(out.substitutions, out.deletions, out.insertions) # 1 1 1
Report WER over a corpus by summing errors and reference words across all utterances, then dividing once. Averaging per-utterance WER gives short utterances outsized weight: one error in a two-word clip counts as 50%.
What WER does not tell you#
- Which words were wrong. A 4% WER that misses every product name is worse for a support team than a 6% WER that only drops filler words. Entity and keyword accuracy measures this directly.
- Whether meaning survived. "gonna" vs "going to" is an error; "can" vs "can't" is also one error. Semantic WER tries to count only errors that change meaning.
- Formatting quality. Punctuation, casing and inverse text normalisation are usually stripped before scoring, yet they decide whether a transcript is readable.
- Speed. A model can trade latency for accuracy. Report WER next to latency or real-time factor.
- Your audio. Public benchmarks are read or broadcast speech, meetings and earnings calls. Your call-centre audio at 8 kHz is none of these. See building your own test set.
How Reson8 approaches this#
Reson8 publishes WER on public sets with the method stated: Resonant-1 averaged 6.51% across the five short-form English sets of the Open ASR Leaderboard, and 4.22% on FLEURS across its European languages excluding Swedish. LibriSpeech and VoxPopuli are left out of the reported average because they were used in training, which would make the score unreliable (see benchmark contamination). Full tables are in the benchmarks post.
Because aggregate WER underweights the words that matter, Reson8 also reports targeted metrics: Patterns is measured on identifier WER (order numbers 8.0% to 1.5%), and the Turns endpoint on semantic WER.
Common questions#
What is a good word error rate?
It depends on the audio. Clean read speech can go below 3%; conversational meetings and phone calls often sit between 8% and 15% for strong systems. The useful question is whether the errors that remain affect your workflow, which a domain test set answers better than a single number.
Can WER be higher than 100%?
Yes. Insertions are counted but the denominator is fixed at the reference length, so a system that outputs many words for a short or silent clip can exceed 100%.
Is accuracy just 1 − WER?
Loosely, and vendors often say "95% accuracy" to mean 5% WER. It is not a true accuracy because insertions can make it negative, so prefer quoting WER directly.
Should I use WER or CER?
WER for languages with clear word boundaries such as English or Spanish. CER for languages without spaces, and as a secondary metric for heavily compounding languages where one wrong word boundary costs several word errors.
References#
- Morris, A. C., Maier, V., & Green, P. (2004). From WER and RIL to MER and WIL: improved evaluation measures for connected speech recognition. Interspeech.
- Radford, A. et al. (2022). Robust Speech Recognition via Large-Scale Weak Supervision (Whisper). arXiv:2212.04356.
- Srivastav, V. et al. Open ASR Leaderboard. Hugging Face.
- Xiong, W. et al. (2016). Achieving Human Parity in Conversational Speech Recognition. arXiv:1610.05256.
- jiwer: evaluate speech recognition with WER, MER, WIL and CER. GitHub.