Speech recognition, also called automatic speech recognition (ASR) or speech-to-text, is the conversion of spoken audio into written text by a machine. A modern system turns the sound wave into a compact time-frequency representation, runs it through a neural network that predicts words or sub-word pieces, and formats the result into readable text. Accuracy is usually measured with word error rate, and it depends at least as much on the audio and the vocabulary as on the model.
Key takeaways
- ASR answers one question: which words were spoken. Identifying who spoke is a separate task, voice recognition.
- Almost every current system follows the same pipeline: audio, features, neural encoder, decoder, text formatting.
- Systems run in two modes: streaming, which emits text while someone is still speaking, and batch, which processes a finished recording.
- The hard cases are conversational speech, noise, accents, overlapping talkers and rare words such as drug names or order codes.
- A single accuracy figure means little without the test set behind it. Performance on your own audio is the number that counts.
Definition#
Speech recognition takes an audio signal and produces the most probable sequence of words that could have generated it. Formally, given acoustic observations X, the system searches for the word sequence W that maximises P(W | X). Older systems split that probability into an acoustic model and a language model; most current systems learn it with one network.
The terms are used loosely. ASR is the research name. Speech-to-text (STT) is the product name, common in APIs. Transcription usually implies a complete, formatted document, often of a recording. Dictation is ASR where the speaker knows a machine is listening and speaks accordingly. All four describe the same core technology.
ASR is distinct from neighbouring tasks. It does not decide who is speaking (diarisation or speaker identification), what the speaker means (natural language understanding), or how to respond (the job of a voice agent). It is the first step that makes those possible.
From sound wave to text#
Every production recogniser, whatever its architecture, moves audio through roughly the same stages.
- Audio capture. Speech is digitised as a stream of samples, typically 16,000 per second for wideband audio or 8,000 for telephone calls. See audio formats.
- Feature extraction. The samples are cut into short overlapping windows, usually 25 ms long every 10 ms, and converted to a log-mel spectrogram: energy per frequency band, on a scale that approximates human hearing.
- Encoding. A neural encoder, today usually a Transformer or Conformer, turns the frames into a sequence of vectors that represent what was said in context.
- Decoding. A decoder converts those vectors into tokens, typically sub-word units, using CTC, a transducer or an attention decoder. This is where search, beam width and context biasing apply.
- Formatting. Raw tokens become readable text: punctuation, casing and inverse text normalisation, which turns "twenty euros" into "€20".
The full mechanics are in how speech-to-text works, and the main decoder families are compared in ASR model architectures.
Streaming and batch#
Recognisers are deployed in two modes with different engineering constraints.
| Streaming (real-time) | Batch (pre-recorded) | |
|---|---|---|
| Input | Audio chunks as they are spoken | A complete file |
| Output | Partial results that are revised, then finals | One finished transcript |
| Context | Past audio, little or no future audio | The whole recording in both directions |
| Main metric besides accuracy | Latency | Throughput and real-time factor |
| Typical uses | Voice agents, live captions, dictation | Call analytics, subtitling, archives |
Because a streaming model cannot wait for the end of the sentence, it has less context and usually a slightly higher error rate on the same audio. Real-time vs batch transcription covers the choice in detail.
What makes speech recognition hard#
Clean read speech is close to solved for major languages. Real audio is not. The errors that remain concentrate in a few places:
- Conversational speech. People restart sentences, trail off, overlap and swallow syllables. Meeting and phone benchmarks produce error rates several times higher than read audiobooks.
- Acoustic conditions. Far-field microphones, reverberant rooms, road noise and narrowband phone codecs remove information the model needs. See noisy audio and telephony transcription.
- Accents and dialects. A model trained mostly on one variety of a language degrades on others. See accents and regional speech.
- Rare words. Proper nouns, product names, drug names and identifiers appear rarely or never in training data, so the model substitutes a common word that sounds similar: metformin becomes met for me. See rare words and structured identifiers.
- Ambiguity. Homophones and word boundaries are resolved only by context: "recognise speech" and "wreck a nice beach" are acoustically close.
Where it is used#
ASR sits underneath a wide range of products. In healthcare it powers dictation and ambient clinical documentation. Contact centres transcribe every call for speech analytics and quality review. Broadcasters generate live captions and subtitles. Courts and law firms transcribe hearings. Cars and phones use it for commands, and voice agents use it as their ears, feeding text to a language model that decides how to reply.
Each setting weights errors differently. A captioning service can tolerate a dropped filler word; a medication list cannot tolerate a wrong dose. That is why evaluation should move beyond WER to the entities and terms that matter in the domain.
How Reson8 approaches this#
Reson8 trains and serves its own model, Resonant-1, on EU GPU infrastructure, covering nine European languages in depth rather than many languages thinly. It offers real-time streaming, batch transcription and a Turns endpoint for voice agents. On the short-form English sets of the Open ASR Leaderboard it averaged 6.51% WER in April 2026, with the method and exclusions stated in the benchmarks post.
For the rare-word problem, Reson8 uses text-only domain adaptation, supplying domain text through the API rather than retraining, and Patterns for identifiers with a known format.
Common questions#
Is speech recognition the same as voice recognition?
No. Speech recognition determines what was said. Voice recognition, or speaker recognition, determines who said it. Consumer products blur the terms, but the models, data and privacy implications differ. See speech vs voice recognition.
How accurate is speech recognition today?
On clean read English, strong systems score around 2% to 3% WER. On meetings, phone calls and accented speech, 8% to 15% is common. Accuracy on specialist vocabulary can be much worse than the headline figure unless the system is adapted to the domain.
Does speech recognition need the internet?
Not necessarily. Small models run on phones and in cars; larger, more accurate models usually run on GPU servers in the cloud or in a private data centre. See on-device vs cloud ASR.
Does a speech recognition system understand what I say?
Only in a narrow sense. It models which word sequences are likely, which captures a lot of grammar and topic, but it does not reason about intent. Understanding is usually handed to a separate language model working on the transcript.
References#
- Jurafsky, D., & Martin, J. H. Speech and Language Processing (3rd ed. draft), chapter on automatic speech recognition. Stanford.
- Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2).
- Hinton, G. et al. (2012). Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 29(6).
- Gulati, A. et al. (2020). Conformer: Convolution-augmented Transformer for Speech Recognition. Interspeech. arXiv:2005.08100.
- Radford, A. et al. (2022). Robust Speech Recognition via Large-Scale Weak Supervision (Whisper). arXiv:2212.04356.
- Srivastav, V. et al. Open ASR Leaderboard. Hugging Face.