The speech recognition wiki
A working reference for people who build with speech: how recognition works, how to measure it, how to make it understand specialist language, and how to run it on European terms.
102
articles
10
sections
364
glossary terms
9
European languages covered
Foundations
Start here. What speech AI is, and how the pieces fit.
- What is speech recognition?ASR, from sound wave to text.
- What is voice AI?Every system that listens, understands or speaks.
- Voice AI vs speech AI vs conversational AIThree overlapping terms, untangled.
- A brief history of speech recognitionFrom Audrey to transformers in seventy years.
- How speech-to-text worksFeatures, encoders, decoders, and text.
- Speech recognition vs voice recognitionWhat was said vs who said it.
- ASR model architecturesCTC, RNN-T, and attention encoder-decoders compared.
- Acoustic models and language modelsThe two halves of recognition, and how they merged.
Speech-to-text
The mechanics of turning audio into reliable transcripts.
- Real-time vs batch transcriptionStreaming and pre-recorded, and when to use each.
- Streaming speech recognitionLive transcripts over a WebSocket.
- Partial vs final resultsWhy live text rewrites itself.
- What is voice activity detection (VAD)?Telling speech apart from silence and noise.
- What is endpoint detection?Deciding when an utterance is over.
- VAD vs endpointing vs turn detectionThree detectors that get confused for each other.
- What is speaker diarization?Who spoke when.
- Diarization vs identification vs verificationRelated speaker tasks with different answers.
- Spoken language identificationDetecting which language is being spoken.
- Code-switching in speech recognitionWhen speakers change language mid-sentence.
- Word timestamps and forced alignmentPlacing every word on the timeline.
- Confidence scores in speech recognitionWhat a 0.87 does and does not tell you.
- Punctuation, casing, and inverse text normalizationFrom 'twenty euros' to €20.
- Hallucinations in speech recognitionWhen a model writes words nobody said.
- Speech-to-text latencyMeasuring the delay that users actually feel.
- Audio formats for speech recognitionSample rates, codecs, and channels.
- Transcribing noisy audioFar-field, overlap, and real-world rooms.
- Telephony transcription8 kHz audio, codecs, and call recordings.
- Long-form transcriptionHours of audio without drift or loss.
- Captions and subtitles: SRT, VTT, and timingTurning transcripts into readable captions.
- Integrating a speech-to-text APIKeys, sessions, WebSockets, and retries.
Accuracy and evaluation
How to measure whether a transcript is good enough.
- What is word error rate (WER)?The standard accuracy metric, explained.
- Speech recognition evaluation beyond WERThe errors a single number hides.
- Semantic word error rateCounting only the errors that change meaning.
- Character error rate (CER)When words are the wrong unit.
- Entity and keyword accuracyMeasuring the words that matter most.
- ASR benchmarks and leaderboardsOpen ASR Leaderboard, FLEURS, and what they test.
- Benchmark contaminationWhy training on test sets inflates scores.
- Building your own ASR test setThe evaluation that predicts production.
- How to evaluate a speech-to-text vendorA practical checklist for buyers.
Domain adaptation
Making a speech model understand your specific language.
- What is domain adaptation in ASR?Why general models fail on specialist speech.
- Fine-tuning vs context biasingRetraining the model vs steering it at runtime.
- Custom vocabulary and context biasingTelling the model which words to expect.
- Text-only domain adaptationAdapting without collecting a single hour of audio.
- Long-context biasingFrom word lists to a million tokens of context.
- Transcribing structured identifiersOrder codes, plates, IBANs, and SKUs.
- Rare words and proper nounsNames, brands, and the long tail.
- Over-biasingWhen custom vocabulary makes accuracy worse.
- Accents and regional speechWhy pronunciation variety breaks models.
- Numbers and alphanumericsDigits, dates, amounts, and spelled-out codes.
European languages
Depth in the languages Europe actually speaks.
- Speech recognition for European languagesDepth over breadth.
- Multilingual speech recognitionOne model, many languages, and the trade-offs.
- Dutch speech recognitionFlemish, compounds, and English loanwords.
- German speech recognitionCompounds, dialects, and case endings.
- French speech recognitionLiaison, homophones, and silent letters.
- Spanish speech recognitionPeninsular and Latin American variants.
- Italian speech recognitionRegional accents and fast speech.
- Polish speech recognitionRich morphology and consonant clusters.
- Portuguese speech recognitionEuropean and Brazilian Portuguese.
- Swedish speech recognitionPitch accent and a smaller data pool.
- European EnglishNon-native English at work.
- Compound words and morphologyWhy 'word' is a slippery unit.
Voice agents
Speech recognition inside AI that talks back.
- What is a voice agent?Software that holds a spoken conversation and acts on it.
- Voice agent architectureHow recognition, reasoning and speech fit together.
- Voice agent latency budgetWhere the milliseconds go.
- End-of-turn detectionKnowing when the caller has finished.
- Turn-taking and barge-inInterruptions, overlaps, and polite agents.
- Speech-to-speech models vs pipelinesOne model or three.
- Voice agent frameworksPipecat, LiveKit Agents, and orchestration.
- Voice agents on the phoneSIP, Twilio, and 8 kHz reality.
- Tool calling in voice agentsWhen the agent needs to look something up.
- Testing voice agentsEvaluating conversations before callers do.
- Multilingual voice agentsSwitching language mid-call.
- What is text-to-speech?The other half of a voice agent.
Industries
How speech recognition is used where the stakes are high.
- Medical speech recognitionClinical vocabulary, dictation, and scribes.
- Ambient clinical documentationFrom consultation audio to a clinical note.
- Legal transcriptionHearings, depositions, and citations.
- Speech recognition in financial servicesTickers, figures, and call recording rules.
- Contact centre speech analyticsEvery call, transcribed and searchable.
- Automotive voice recognitionRoad noise, commands, and in-car assistants.
- Media and broadcast transcriptionLive captions, archives, and subtitling.
- Speech recognition in the public sectorCouncil meetings, parliaments, and citizen services.
Privacy, security, and compliance
Handling voice data in regulated environments.
- GDPR and voice dataWhen audio becomes personal data.
- EU data residency for speech AIWhere audio is processed, and why it matters.
- Zero data retentionProcessing audio without keeping it.
- Data processing agreementsWhat a DPA under GDPR Article 28 covers.
- The EU AI Act and speech recognitionRisk tiers and obligations for voice systems.
- ISO 27001 for speech AI vendorsWhat the certification does and does not prove.
- On-premise and VPC deploymentRunning speech recognition inside your perimeter.
- PII redaction in transcriptsRemoving personal data from text and audio.
- Voice biometrics and consentWhen a voice identifies a person.
- Sovereign AI in EuropeControl over models, data, and infrastructure.
Audio intelligence
What happens after the transcript.
- What is audio intelligence?Extracting meaning from recorded conversations.
- Conversation summarizationFrom an hour of talk to five lines that matter.
- Sentiment analysis on speechWords, tone, and what they miss.
- Keyword spotting and wake wordsSmall models that listen for one phrase.
- Audio event detectionRecognising laughter, alarms, music and silence.
- Using transcripts with LLMsFeeding speech into language models.
Infrastructure and cost
Serving speech models fast, at scale, and within budget.
- Real-time factor (RTF and RTFx)How fast a model processes audio.
- GPU inference for speech modelsKernels, batching, and CUDA graphs.
- Concurrency and scalingThousands of streams at once.
- Model size and quantizationSmaller, faster models and what they cost in accuracy.
- On-device vs cloud speech recognitionWhere the model runs, and the trade-offs.
- Speech-to-text pricing modelsPer minute, credits, and hidden costs.