Rare words and proper nouns are where general-purpose speech recognition fails most visibly. A model can transcribe everyday speech at a low error rate and still misspell the customer's surname, the drug being prescribed and the street being named, because those words sit in the long tail of language and appeared rarely or never in training. The fix is to give the model the names it will hear, and to measure name accuracy separately from overall word error rate.
Key takeaways
- Word frequency follows a steep curve. A small set of common words covers most running speech; most distinct words are rare.
- Names carry the meaning in business audio: who, which product, which account. They are also the words models know least.
- Modern recognisers can spell unseen words from subword units, but with a weak prior they spell them as the nearest common words.
- Some names are ambiguous from audio alone. Jansen and Janssen sound identical; only context or a record can decide.
- Track entity recall and precision; aggregate WER hides name errors because names are a small share of words.
The long tail#
In any large text collection, a word's frequency falls roughly in inverse proportion to its frequency rank, a regularity known as Zipf's law. The consequence for speech recognition is that the few hundred most common words account for a large share of everything spoken, while tens of thousands of words each occur only a handful of times.
Training data amplifies the effect. A model trained on podcasts and broadcast audio has heard "Amsterdam" many thousands of times and "Hoogkarspel" perhaps never. It has heard "paracetamol" but may not know "levetiracetam". The first set of words has a strong prior; the second has almost none.
Why models get names wrong#
No prior, or the wrong one#
End-to-end recognisers use subword units, so there is no fixed vocabulary and no hard "out of vocabulary" failure. A model can in principle spell any word. In practice, when the internal language model has little evidence for a rare spelling, the decoder prefers a sequence of common subwords that sounds similar. The rare word is split, respelled or replaced.
Spelling is not recoverable from sound#
For common words the model has learned the spelling. For names, the spelling is often arbitrary. "Thijs", "Tijs" and "Ties" are all Dutch first names that can be pronounced alike. English has "Catherine", "Kathryn" and "Katharine". The audio cannot say which one the speaker means.
Cross-language names#
Names travel between languages and are pronounced with the speaker's phonology. A French speaker saying a Polish surname, or an English speaker saying a Dutch street name, produces sounds the model associates with neither language's spelling. See code-switching and accents and regional speech.
Typical errors#
| Error type | Said | Heard |
|---|---|---|
| Split into common words | levetiracetam | leave it here a set am |
| Phonetic respelling | Keizersgracht | Kaiser's Gracht |
| Normalised to a common variant | Janssen | Jansen |
| Place name split and respelled | Hoogkarspel | Hoog Kaspel |
| Brand stylisation lost | iPhone 15 Pro | I phone fifteen pro |
Each of these is one or two errors in WER terms. In a support ticket, a patient record or a court transcript, each can make the record wrong.
Ways to fix it#
Give the model the names in advance#
The most direct fix is to tell the recogniser which rare words to expect. Short lists of names work through custom vocabulary and context biasing. Larger sets, such as a formulary or a product catalogue, work better as text in sentences through text-only domain adaptation or long-context biasing, because surrounding words tell the model when each name is likely.
Make the context specific to the conversation#
A contact centre with 200,000 customers should not bias every call toward every surname. If the caller has been identified, the relevant names are the ones on that account: their name, their products, their recent orders. Narrow, per-session context tends to keep the gain while cutting distractors sharply. The risks of large, generic lists are covered in over-biasing.
Correct after recognition#
Where the set of valid names is known, a post-processor can match transcript spans against it using fuzzy string matching or phonetic keys such as Double Metaphone. This catches respellings like "Kaiser's Gracht" but cannot recover information the recogniser discarded, and it can introduce errors when a real, unlisted name is snapped to a listed one.
Ask for the spelling#
In a voice agent, where a name drives an action, asking the caller to spell it, or confirming it by reading it back, resolves ambiguities that no model can. Spelled names are then a structured identifier problem.
Measuring name accuracy#
Because names are a small fraction of spoken words, a system can halve its name errors and move aggregate WER by a fraction of a point. Measure names directly:
- Entity recall: of the names in the reference, how many appear correctly spelled in the transcript.
- Entity precision: of the names in the transcript, how many were actually said. This catches false insertions from biasing.
- Per-category breakdown: people, places, products and clinical terms fail in different ways.
Decide in advance what counts as correct. "Janssen" for "Jansen" is an error if you are matching records, and acceptable if you only need searchable transcripts. Entity and keyword accuracy covers scoring in more depth.
How Reson8 approaches this#
Reson8 handles rare terms through text-only domain adaptation: you supply domain text through the API, with no audio, fine-tuning or retraining, and it takes effect in under 60 seconds. Up to 1M tokens of context is applied at runtime without latency cost, and it supports 50,000+ medical terms with no latency impact. Reson8's founders previously built large-scale medical transcription platforms, where drug and procedure names are the long tail that matters most.
Adaptation does not resolve names that are ambiguous from sound alone; a record or a confirmation step is still needed for those. See the documentation for setup.
Common questions#
Why does the model get a famous name right but not my customer's?
Famous names appear often in training data, so the model has a strong prior for their spelling. Your customer's name probably never appeared, so the model falls back on common words that sound similar.
Can I just add every customer name to a vocabulary list?
For small lists, yes. For very large lists, many entries will sound like ordinary words or like each other, and accuracy on the rest of the transcript can drop. Scope the list to the conversation where you can.
Should I fix names with an LLM after transcription?
An LLM with access to the right records can repair some errors, but it cannot hear the audio and may substitute a plausible wrong name. Biasing during recognition keeps the audio evidence in the decision.
References#
- Zipf, G. K. (1949). Human Behavior and the Principle of Least Effort. Addison-Wesley.
- Sennrich, R., Haddow, B., & Birch, A. (2016). Neural Machine Translation of Rare Words with Subword Units. ACL.
- Kudo, T., & Richardson, J. (2018). SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing. EMNLP (System Demonstrations).
- Pundak, G. et al. (2018). Deep Context: End-to-End Contextual Speech Recognition. IEEE SLT.
- Philips, L. (2000). The Double Metaphone Search Algorithm. C/C++ Users Journal, 18(6).