Text-only domain adaptation improves a speech recognition model on specialist language using written text alone: glossaries, product catalogues, clinical notes, past reports. It works because most domain errors are not acoustic failures but failures of expectation. The model hears the sounds correctly and then picks a common word over the rare one it has never seen in context. Supplying domain text fixes the expectation without collecting or transcribing a single hour of audio.
Key takeaways
- Most domain errors come from the language side of the model. A general model has the acoustics right and the priors wrong.
- Text is cheap and already exists in most organisations. Transcribed domain audio is slow, expensive and often cannot leave the building.
- Techniques range from external language models fused at decode time to runtime context that the model reads with the audio.
- Text cannot fix acoustic mismatch. Heavy noise, 8 kHz telephony and unfamiliar accents still need acoustic robustness.
- Measure on held-out domain audio and on ordinary speech, so you catch gains on terms and losses elsewhere.
Why text is enough for most domain errors#
An end-to-end recogniser learns two things at once from its training data: how speech sounds map to units of text, and which sequences of words are likely. The second part is often called the internal language model. It is shaped by the transcripts the model was trained on, which for general-purpose systems means podcasts, audiobooks, meetings, broadcast and web video.
When a cardiologist says "start apixaban 5 milligrams", the acoustic evidence for apixaban is usually clear. The problem is that the internal language model has seen the word rarely or never, so a phonetically close sequence of common words wins the search:
start apixaban five milligrams vs start a pick a ban five milligrams
Nothing about this error requires new audio to fix. The model needs to know that apixaban is a word, how it is spelled, and that it follows "start" and precedes a dose. All three facts are present in ordinary written text from the domain. This is the core argument for text-only adaptation, and the reason domain adaptation in speech increasingly starts with text rather than recordings.
Methods#
Text-only approaches differ in where the domain knowledge enters the system and whether the model's weights change.
External language model fusion#
The oldest approach. Train an n-gram or neural language model on domain text and combine its scores with the ASR model's scores during beam search. This is shallow fusion, carried over to speech from neural machine translation. It is simple and needs no change to the acoustic model, but the fusion weight has to be tuned, and the domain LM can pull the decoder toward in-domain words even when the audio disagrees.
Correcting for the internal language model#
Because the end-to-end model already carries a general-domain language model, adding a domain LM on top double-counts some evidence. Density ratio methods and internal language model estimation (ILME) subtract an estimate of the source-domain prior before adding the target-domain one. Both have been shown to give larger gains than plain shallow fusion when source and target domains differ sharply.
Adapting part of the network on text#
In transducer models (RNN-T), the prediction network behaves like a language model over previous output tokens. It can be fine-tuned on domain text while the acoustic encoder stays frozen. This changes weights, so it needs a training run and a new model version per domain, but it needs no audio.
Synthetic audio#
Another route converts domain text into speech with a text-to-speech system and fine-tunes on the result. It gives the model audio-text pairs for rare words, though synthetic voices are cleaner and less varied than real speakers, and models can learn TTS artefacts. It is usually combined with real audio rather than used alone.
Runtime context#
The most recent approach supplies domain text to the model at inference time, as context it attends to while decoding. No weights change and no model is retrained. Early forms were short phrase lists (see custom vocabulary and context biasing); newer systems accept whole documents, covered in long-context biasing.
| Method | Changes weights? | Turnaround | Main risk |
|---|---|---|---|
| Shallow fusion with domain LM | No (separate LM trained) | Hours to train the LM | Fusion weight too high pulls in wrong terms |
| ILME / density ratio | No | Hours | Needs a good estimate of the source prior |
| Prediction network fine-tuning | Yes | Hours to days | One model version per domain to maintain |
| TTS-augmented fine-tuning | Yes | Days | Model learns synthetic voice artefacts |
| Runtime context | No | Seconds to minutes | Poorly chosen context can cause over-biasing |
What text to supply#
The best domain text looks like what people say, not only what they write. A few rules hold across methods:
- Terms in sentences beat bare lists. "Continue apixaban 5 mg twice daily" teaches spelling, word order and neighbouring words. "apixaban" on its own line teaches only spelling.
- Include the spoken forms of things that are written differently. If staff say "the Q3 deck" and write "Q3 presentation", include both.
- Prefer your own documents. Past reports, ticket histories, internal wikis and product catalogues match your terminology better than public glossaries.
- Remove what should not be recognised. Old product names, boilerplate footers and legal disclaimers add prior mass for words nobody will say.
- Handle personal data like any other data. Clinical notes and call logs used as adaptation text can contain personal data; apply the same controls as for the audio. See GDPR and voice data.
Limits of text-only adaptation#
Text shifts the model's expectations. It does not change what the model can hear. Several failure types are outside its reach:
- Acoustic mismatch. Far-field microphones, overlapping speakers and 8 kHz telephone audio degrade the acoustic evidence itself. See transcribing noisy audio.
- Unusual pronunciations. If a brand is pronounced nothing like its spelling suggests, the model has to guess the mapping. Text alone gives it the spelling, not the sound. Accent effects are covered in accents and regional speech.
- Ambiguity the audio cannot resolve. Jansen and Janssen sound the same. Text can make one more likely; only an external record can say which is right.
- Formats without a lexicon. Order codes and licence plates have structure but no dictionary. They need format constraints rather than word lists; see structured identifiers.
Measuring the effect#
Adaptation changes the output distribution, so evaluate both what it is meant to fix and what it might break. Build two slices from real domain audio, as described in building your own ASR test set:
- Utterances containing domain terms. Report term recall and precision alongside WER, because a handful of fixed drug names barely moves aggregate WER.
- Utterances without domain terms. WER here should stay flat. If it rises, the adaptation is inserting domain words into ordinary speech.
Keep the audio fixed and vary only the adaptation text. Because runtime methods take effect without retraining, this loop can run many times a day.
How Reson8 approaches this#
Reson8's domain adaptation is text-only. You supply domain text through the API, and it takes effect in under 60 seconds with no fine-tuning, no audio data and no retraining. Up to 1M tokens of context is applied at runtime without latency cost, and adaptation supports 50,000+ medical terms with no latency impact. For identifiers that follow a known format, Patterns complements text adaptation.
The acoustic limits above apply to Reson8 as to any system: domain text will not recover words the audio does not support. Setup details are in the documentation.
Common questions#
Do I need any audio at all?
Not to adapt. You do need a modest set of real, transcribed domain recordings to measure whether the adaptation helped. A few hours is usually enough to see term-level differences.
How much text is enough?
It depends on the method. Runtime context can help with a single page of terms in sentences. Training a separate domain language model typically benefits from far more text, in the millions of words.
Is this the same as fine-tuning?
No. Fine-tuning updates the model's weights, usually on audio. Runtime text adaptation leaves the weights alone. The trade-offs are covered in fine-tuning vs context biasing.
Can adaptation make accuracy worse?
Yes, if the text pulls domain words into ordinary speech. This is over-biasing, and it is why you should test on utterances without domain terms too.
References#
- Gulcehre, C. et al. (2015). On Using Monolingual Corpora in Neural Machine Translation. arXiv:1503.03535.
- Kannan, A. et al. (2018). An Analysis of Incorporating an External Language Model into a Sequence-to-Sequence Model. ICASSP.
- McDermott, E., Sak, H., & Variani, E. (2019). A Density Ratio Approach to Language Model Fusion in End-to-End Automatic Speech Recognition. IEEE ASRU.
- Meng, Z. et al. (2021). Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition. IEEE SLT.
- Pylkkönen, J. et al. (2021). Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Network. Interspeech.
- Rosenberg, A. et al. (2019). Speech Recognition with Augmented Synthesized Speech. IEEE ASRU.