Long-context biasing steers a speech recogniser with large amounts of domain text, from a few pages up to hundreds of thousands of words, rather than a short list of keywords. Classic biasing methods break down as lists grow, because every extra entry is a potential distractor and often a latency cost. Long-context approaches treat the text as evidence about the domain, letting the model use spelling, word order and co-occurrence instead of a flat list of boosted strings.
Key takeaways
- Early biasing systems accepted tens to thousands of phrases. Real domains have tens of thousands of terms.
- Bigger bias lists bring more distractors: entries that sound like ordinary speech and get inserted where they were not said.
- Context in sentences carries information that lists cannot, such as which dose follows which drug.
- Scale is only useful if latency stays flat. Methods that attend over every entry per frame get slower as context grows.
- Test long context on utterances without domain terms, not only on the terms you added.
From word lists to context#
The first generation of custom vocabulary features took a list of phrases and a boost weight. Under the hood, the phrases were compiled into a small grammar or prefix tree, and hypotheses that matched a phrase received a score bonus during beam search. For a voice assistant that needs to find a contact name, a list of a few hundred names works well.
Specialist domains are larger. A hospital formulary, a bank's product catalogue or a manufacturer's parts list can run to tens of thousands of entries. Adding them all to a flat list causes three problems:
- Distractors. The more entries, the more likely one sounds close to ordinary speech. Each one is a chance to insert a term nobody said. See over-biasing.
- Lost context. A list says a term exists. It does not say when it is likely. "Metoprolol" is plausible after "start" and implausible after "the meeting is at".
- Cost. Neural biasing methods that attend over every phrase at every decoding step scale with the size of the list, so latency rises with context.
How biasing mechanisms work#
Three families of technique are in common use. They can be combined.
Shallow fusion on a prefix tree#
Bias phrases are stored in a trie of subword units. As the beam search extends a hypothesis, tokens that continue a trie path receive a bonus, and the bonus is withdrawn if the path is abandoned. This is cheap and works with any decoder, but the boost is a single number applied without regard to context.
Attention-based contextual biasing#
Contextual Listen, Attend and Spell (CLAS) and later deep biasing models encode each bias phrase into an embedding. The decoder attends over those embeddings, including a learned "no bias" option, and decides at each step whether any phrase is relevant. This is learned end to end and handles context better, but attention cost grows with the number of phrases.
Prompting with text#
Models with a text decoder can be given preceding text as a prompt. Whisper, for example, accepts an initial prompt, but only its final 224 tokens are used. The prompt shifts style and vocabulary, and it works best when written as natural sentences. Long-context biasing extends this idea from a few hundred tokens to far larger inputs.
Why more context can help rather than hurt#
If long context only meant longer lists, it would make over-biasing worse. The gain comes from treating the input as text rather than as entries. Written context tells the model:
- Collocations. Which terms appear next to which. A dose unit after a drug name, a part number after "replace".
- Spelling conventions. Whether your organisation writes "e-mail" or "email", "Covid-19" or "COVID-19".
- Relative frequency. A term that appears two hundred times in the context is more likely than one that appears once.
- What is absent. A context that never mentions a common word does not suppress it, but a context dense with one sense of an ambiguous word shifts the choice toward that sense.
An example from a maintenance call:
swap the MX-400 bracket on bay two vs swap the MX four hundred racket on bay two
A list containing "MX-400" fixes the part name. A context of maintenance manuals also makes "bracket" more likely than "racket" in the same sentence, which a list of part numbers cannot do.
What fits in a given context#
Token counts depend on the tokeniser. For English prose with a typical subword tokeniser, a rough rule is three quarters of a word per token; languages with long compounds, such as Dutch and German, use more tokens per word.
| Context size | Roughly equivalent to | Typical use |
|---|---|---|
| Hundreds of tokens | A list of names or a short paragraph | Contact lists, meeting attendee names |
| Thousands of tokens | A glossary of a few pages | A single product line, a team's jargon |
| Around 100,000 tokens | A long handbook | A formulary, a full parts catalogue |
| Around 1,000,000 tokens | Several thousand pages of text | Knowledge bases, archives of past reports |
Latency and freshness#
For streaming recognition and voice agents, biasing that adds tens of milliseconds per utterance is a real cost; see voice agent latency budget. When evaluating a long-context feature, check three numbers at your intended context size, not at the vendor's demo size:
- Time to first partial and to final result, compared with no context.
- How long an update to the context takes to become active.
- Whether the context is fixed per account, per session, or per request, since that decides whether you can supply a patient's own medication list for one consultation.
How Reson8 approaches this#
Reson8's domain adaptation accepts up to 1M tokens of domain text, applied at runtime without latency cost, and supports 50,000+ medical terms with no latency impact. Adaptation needs no fine-tuning, audio or retraining, and takes effect in under 60 seconds, so the measure-and-edit loop described above can run within a working session.
The same caution applies as with any biasing system: evaluate on your own audio, including utterances that contain none of the supplied terms. Integration details are in the documentation.
Common questions#
Is long-context biasing the same as an LLM correcting the transcript afterwards?
No. Post-correction with a language model rewrites text after recognition and can only work from what the recogniser produced. Biasing acts during decoding, while the audio evidence is still available to choose between candidates.
Should I paste my whole document store in?
Start smaller. Documents that match how people speak in your calls help most. Boilerplate, legal footers and outdated product names add noise.
Does long context replace fine-tuning?
For vocabulary and phrasing, often yes. For acoustic problems such as unusual microphones or accents, no. See fine-tuning vs context biasing.
How do I measure whether it worked?
Track term recall on utterances containing your terms and WER on utterances without them. Entity and keyword accuracy explains the metrics.
References#
- Aleksic, P. et al. (2015). Bringing Contextual Information to Google Speech Recognition. Interspeech.
- Pundak, G., Sainath, T. N., Prabhavalkar, R., Kannan, A., & Zhao, D. (2018). Deep Context: End-to-End Contextual Speech Recognition. IEEE SLT.
- Zhao, D. et al. (2019). Shallow-Fusion End-to-End Contextual Biasing. Interspeech.
- Jain, M. et al. (2020). Contextual RNN-T for Open Domain ASR. Interspeech.
- Le, D. et al. (2021). Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion. Interspeech.
- OpenAI. Speech to text guide: prompting. OpenAI documentation.