Over-biasing is what happens when custom vocabulary or context biasing pushes a recogniser too hard toward the supplied terms. The model starts writing those terms where the speaker said something else that sounded similar. Accuracy on the target words goes up while accuracy on everything else goes down, and if you only test on the target words you will not see it. The cure is to scope and word the bias carefully, and to always measure on speech that contains none of the bias terms.
Key takeaways
- Biasing trades missed terms for false insertions. Past a point, every extra unit of bias costs more than it gains.
- The main causes are high boost weights, large generic lists, short or common-word entries, and terms that sound like everyday phrases.
- Over-biasing is invisible on a test set built only from utterances containing the terms.
- Measure term precision as well as term recall, and WER on an anti-context set with no bias terms.
- Scoped, per-session context and multi-word phrases are the most reliable fixes.
What it looks like#
A clinic adds its formulary to the recogniser's context. Drug names improve. Then this appears in a consultation transcript:
we'll keep her on it for now vs we'll Keppra on it for now
"Keppra" is a real brand name for levetiracetam, and in the right context it should be recognised. Here it was not said. The acoustic evidence for "keep her" and "Keppra" is close, and the bias tipped the decision the wrong way.
A few other forms are common:
- Brand names that are also words. A software company biases toward its product "Clarity" and gets Clarity for clearly.
- Names that are also everyday words. A contact list containing Will, May and Rose turns ordinary verbs and nouns into capitalised names.
- Formats applied too broadly. An order-number format active for a whole call turns for two into a code fragment in an unrelated sentence.
Why it happens#
Every biasing method adds a score to hypotheses that match the context. Recognition is a competition between hypotheses, so any boost to one is a relative penalty to the others. When the audio clearly supports the ordinary words, a modest boost does no harm. When the audio is ambiguous, as it often is in fast or noisy speech, the boost decides.
The risk grows with four factors:
- Boost weight. In shallow-fusion biasing, a fixed bonus is added for matching tokens. Too high, and matching hypotheses win against clear acoustic evidence.
- List size. Each entry is a potential distractor in every utterance where it is not spoken. Research on contextual models consistently reports that accuracy degrades as the number of distractor phrases grows.
- Entry length. Short entries, especially single syllables, match fragments of ordinary speech. Multi-word phrases are much harder to trigger by accident.
- Phonetic neighbourhood. Entries close in sound to frequent words or phrases, like "Keppra" and "keep her", are the most dangerous regardless of weight.
Diagnosing over-biasing#
| Symptom | Likely cause | First fix to try |
|---|---|---|
| A bias term appears in unrelated sentences | Entry sounds like a common phrase | Replace it with a longer phrase that includes context |
| WER on general speech rose after adding a list | List too large or too generic | Scope the list to the session or account |
| Short function words replaced by names | Single-syllable entries | Remove them, or include first name plus surname |
| Codes appear inside ordinary sentences | Format active outside the turn that needs it | Enable the format only when a code is expected |
| Correct term, wrong spelling variant forced | Two variants in context, one dominant | Keep only the variant you want written |
Measuring it#
Most biasing evaluations report recall on utterances containing the bias terms. That measures only the left half of the diagram. A complete evaluation, as described in building your own ASR test set, has three parts:
- Term set. Utterances where bias terms were spoken. Report term recall and overall WER.
- Anti-context set. Utterances from the same domain containing none of the bias terms. Run them with the bias active. WER should match the unbiased baseline.
- Term precision. Across both sets, of all bias terms in the output, how many were actually said. This is the most direct measure of over-biasing. See entity and keyword accuracy.
Run the three measurements at more than one context size. A list that behaves at 200 entries may not at 20,000.
Avoiding it#
Scope the context#
Supply the terms that are likely in this conversation rather than every term the organisation uses. A caller's own products, a patient's own medication list, the parties to one legal matter. Per-session context can cut the distractor count by orders of magnitude.
Write phrases, not fragments#
"Keppra 500 milligrams" is much harder to trigger by accident than "Keppra". Terms in natural sentences, as used in text-only domain adaptation and long-context biasing, carry the context that tells the model when a term fits.
Prune common words#
Remove entries that are ordinary words in the language of the audio, unless you have evidence they are misrecognised. A bias list is for words the model gets wrong, not for every word you care about.
Tune strength where you can#
If the system exposes a weight, sweep it on the three test sets above and pick the value where term recall stops improving faster than term precision falls.
Restrict formats in time#
For structured identifiers, apply the expected format on the turn where the identifier is requested, not for the whole call.
How Reson8 approaches this#
Reson8's domain adaptation takes domain text rather than requiring retraining, and changes take effect in under 60 seconds. That makes the test, edit and re-test loop above practical within a working session. Patterns results are published on identifier WER; as with any biasing, check anti-context speech on your own audio before rollout. See the documentation.
Common questions#
Is over-biasing the same as hallucination?
They overlap. Both produce words that were not said. Hallucinations usually come from the model's own priors, often during silence or noise; over-biasing comes from context you supplied, and it tends to produce the specific terms in that context.
How big can a bias list get before it hurts?
There is no universal number. It depends on the method, how confusable the entries are and the audio. Measure on an anti-context set at your real list size.
Should I remove a term that causes false insertions?
Try rewriting it as a longer phrase first. If it still fires on ordinary speech and the model recognised it acceptably without bias, remove it.
Does per-session context add latency?
It depends on the system. Check time to final result with and without context at the size you intend to use. See speech-to-text latency.
References#
- Aleksic, P. et al. (2015). Bringing Contextual Information to Google Speech Recognition. Interspeech.
- Pundak, G., Sainath, T. N., Prabhavalkar, R., Kannan, A., & Zhao, D. (2018). Deep Context: End-to-End Contextual Speech Recognition. IEEE SLT.
- Alon, U., Pundak, G., & Sainath, T. N. (2019). Contextual Speech Recognition with Difficult Negative Training Examples. ICASSP.
- Zhao, D. et al. (2019). Shallow-Fusion End-to-End Contextual Biasing. Interspeech.
- Le, D. et al. (2021). Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion. Interspeech.