Speech recognition is usually scored in words, but a "word" means different things in different languages. German, Dutch and Swedish write compounds as one word, Finnish and Hungarian attach several suffixes to a stem, and Polish changes the ending of nearly every noun by case. These habits enlarge the vocabulary a model must produce, create spelling choices at word boundaries, and make word error rate count some small mistakes twice.
Key takeaways
- In compounding languages, one boundary error costs two WER errors: splitting Rechtsschutzversicherung into two words is a substitution plus an insertion.
- Rich morphology multiplies the number of distinct word forms, so each form is seen less often in training and more words behave like rare words.
- Modern ASR models output subword tokens, which removes the hard out-of-vocabulary limit but not the boundary decision.
- Report character error rate or compound-split WER alongside plain WER for German, Dutch, Swedish, Finnish and similar languages.
- Normalisation rules for hyphens, spaces and apostrophes can move scores as much as model differences do.
What counts as a word#
For scoring, a word is whatever sits between spaces after normalisation. That definition works tolerably for English, where compounds are mostly written open (health insurance, customer service), and breaks down elsewhere. Linguists distinguish several things that orthography blurs:
- Compounding: joining stems into a new word. German Kranken + Haus gives Krankenhaus (hospital); Dutch zorg + verzekering gives zorgverzekering (health insurance); Swedish sjuk + sköterska gives sjuksköterska (nurse).
- Inflection: changing a word's form for grammar. Polish klient, klienta, klientowi, klientem and more are all "customer".
- Agglutination: stacking suffixes, each with one meaning. Finnish talo (house), talossa (in the house), talossani (in my house), talossanikin (in my house too). Turkish evlerinizden is "from your houses".
- Cliticisation: attaching short pronouns or particles in writing. Italian dammelo (give it to me), Spanish dámelo, French l'hôpital.
Each of these makes the space-delimited "word" a larger and more variable unit than in English, and each has consequences for recognition and for scoring.
How compounds distort WER#
Word error rate penalises a boundary mistake more heavily than a reader would. If a model writes a German compound as two words, the alignment finds one substitution and one insertion against a single reference word:
On a full test set the effect is diluted, but it is systematic. A German or Dutch system that splits compounds more often than another will show a higher WER even when the two produce equally readable transcripts. The reverse error, merging two words that should be separate (zu Hause as zuhause), is penalised the same way.
Scoring options#
- Character error rate: counts edits over characters, so a split compound costs one inserted space. It gives partial credit to near-miss inflections too, which suits Polish or Finnish. See CER.
- Compound-split WER: run the same compound splitter over reference and hypothesis before scoring, so both sides use the same segmentation. Corpus-based splitting methods such as Koehn and Knight's, originally for machine translation, work well for this.
- Space-insensitive matching: remove spaces inside candidate compounds when the joined form exists in a lexicon.
- Morph-level scoring: segment both sides into morphs, for example with Morfessor, and score over those. An option for agglutinative languages such as Finnish and Turkish.
Vocabulary size and data sparsity#
Compounding and inflection both increase the number of distinct word forms. A productive compounding language has an effectively open vocabulary: speakers form Zahnarztterminerinnerung (dentist appointment reminder) on the spot, and it will never appear in any training corpus as a whole. A richly inflected language multiplies each lemma into many forms, and the rarer forms of common words appear only a handful of times in training data.
Older systems with a fixed word vocabulary simply could not output words they had not seen. These out-of-vocabulary words were a large share of errors in German, Finnish and Turkish ASR, and research in the 2000s addressed them with morph-based language models built on units discovered by tools such as Morfessor.
Modern end-to-end models avoid the hard limit by predicting subword tokens learned with byte-pair encoding or a unigram model, typically through SentencePiece. A new compound can be spelled out from familiar pieces. What remains is a soft version of the same problem: the model must decide where spaces go, rare forms are still less probable, and a token inventory learned mostly from English may cut German or Polish words at unhelpful points, making sequences longer and slower to decode. This is one reason tokeniser design matters so much for multilingual models.
How it plays out by language#
| Language | Main feature | Typical error |
|---|---|---|
| German | Closed compounds, case endings | Compound split; dem/den swapped |
| Dutch | Closed compounds, linking -s- and -en- | zorg verzekering; pannekoek for pannenkoek |
| Swedish | Compounds, where splitting changes meaning | sjuk sköterska (sick nurse) for sjuksköterska (nurse) |
| Polish | Seven cases, inflected numerals | Right stem, wrong ending |
| Finnish | Agglutination, fifteen cases | Missing or wrong suffix on a long word |
| English | Mostly open compounds, spelling variants | healthcare vs health care vs health-care |
English is not immune. Email, e-mail; login, log-in, log in; data set, dataset: all are live variants, and scoring inconsistency between them is a common source of false errors in English WER. For language-specific detail, see German, Dutch, Swedish and Polish.
How Reson8 approaches this#
Three of the languages Resonant-1 supports are heavily compounding: German (3.83% on FLEURS), Dutch (4.88%) and Swedish (7.38%, its weakest published figure), and Polish (4.56%) is richly inflected. Figures are in the benchmarks post. Domain-specific compounds, such as product names or clinical terms, can be supplied as text through domain adaptation, which takes effect in under 60 seconds without audio or retraining.
Common questions#
Why does my German WER look worse than my English WER on similar audio?
Partly compounding: each split or merge costs two errors. Partly inflection: case endings are short and easy to miss. Check CER or compound-split WER before concluding the model is worse.
Can a model output a compound it has never seen?
Yes, if it uses subword tokens, which nearly all current models do. Whether it writes the compound joined or split depends on what its training data taught it about that pattern.
Should transcripts split long compounds for readability?
No. Readers of German, Dutch and Swedish expect closed compounds, and in Swedish splitting can change the meaning. Follow the language's spelling norms.
References#
- Koehn, P., & Knight, K. (2003). Empirical Methods for Compound Splitting. EACL 2003.
- Creutz, M., & Lagus, K. (2007). Unsupervised models for morpheme segmentation and morphology learning. ACM Transactions on Speech and Language Processing, 4(1).
- Hirsimäki, T. et al. (2006). Unlimited vocabulary speech recognition with morph language models applied to Finnish. Computer Speech & Language, 20(4).
- Sennrich, R., Haddow, B., & Birch, A. (2016). Neural Machine Translation of Rare Words with Subword Units. ACL 2016.
- Kudo, T., & Richardson, J. (2018). SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. EMNLP 2018 (System Demonstrations).