Keyword spotting detects when a specific word or short phrase is spoken in a continuous stream of audio, without transcribing everything else. Its best-known use is the wake word ("Hey Siri", "Alexa", "OK Google") that makes a device start listening properly. Wake-word models are tiny, run on the device all the time, and are tuned for one trade-off above all: how often they wake by mistake versus how often they miss the user. A second meaning, searching recordings for terms, is now usually done by transcribing the audio in full and searching the text.
Key takeaways
- Wake-word detectors are small classifiers, often well under a million parameters, running continuously on a low-power chip or CPU.
- They are evaluated by false rejects (missed wake-ups) and false accepts per hour (spurious wake-ups). Improving one usually worsens the other.
- A detection only opens the gate. Full speech recognition, on the device or in the cloud, handles the request that follows.
- Good wake words are several syllables long, phonetically distinct and rare in normal conversation.
- For searching recorded calls for terms, full transcription plus text search is simpler and more flexible than dedicated keyword spotting.
Two meanings of keyword spotting#
Wake words and commands. A device listens for one phrase, or a small set such as "stop", "next" and "volume up", and acts when it hears it. The vocabulary is fixed in advance and the model is trained for exactly those phrases.
Spoken term detection. Given a large archive of recordings, find every place a term was said. This was an active research area when full transcription was expensive or unavailable for a language, and programmes such as IARPA Babel studied it for low-resource languages. For well-supported languages, transcribing everything with a general ASR model and searching the text is now the usual approach. Its accuracy is then a question of entity and keyword accuracy rather than a separate detector.
The rest of this article focuses on the first meaning.
The always-listening cascade#
Running a full speech recogniser continuously would drain a battery and send every sound in the room to a server. Devices instead use a cascade, where each stage is more expensive and runs less often than the one before.
- Voice activity detection. A very cheap detector, sometimes in dedicated hardware, decides whether there is sound resembling speech at all. See voice activity detection.
- Wake-word model. A small neural network scores short overlapping windows of audio, typically every 10 to 30 milliseconds, for the probability that the wake phrase just ended.
- Verification. Many products run a larger second-stage model, on the device or on a server, on the buffered audio to reject false triggers before the assistant responds.
- Full recognition. The request after the wake word ("what's the weather tomorrow") is transcribed by a full model, and the assistant takes over.
How wake-word models work#
Most detectors take log-mel filterbank features or MFCCs over a sliding window of around one second and output a score per frame. Architectures have moved from small hidden Markov models and fully connected networks to convolutional networks, depthwise-separable CNNs and small attention models, all chosen for low memory and multiply count.
Chen, Parada and Heigold (2014) described a small-footprint deep neural network approach for Google's on-device keyword spotting, and Sainath and Parada (2015) showed that convolutional networks reduced false rejects at the same footprint. Zhang et al. (2017), in "Hello Edge", benchmarked several architectures on microcontrollers with a few hundred kilobytes of memory. The Google Speech Commands dataset (Warden, 2018), with about 105,000 one-second utterances of 35 words, became the standard public benchmark for small-vocabulary keyword spotting.
Training data is the hard part. A good detector needs thousands of speakers saying the phrase in different accents, rooms and distances, and far more hours of negative audio: television, music, conversation, and words that sound similar. Synthetic speech from text-to-speech systems is increasingly used to generate positive examples for new phrases, with open-source projects such as openWakeWord built around that approach.
Measuring wake-word performance#
Two error types matter, and they are reported on different scales:
| Metric | Definition | User experience |
|---|---|---|
| False reject rate (FRR) | Missed detections ÷ times the phrase was spoken | "I have to say it twice." |
| False accepts per hour (FA/h) | Spurious detections ÷ hours of audio without the phrase | "It wakes up during films." |
| Detection latency | Time from end of phrase to trigger | Perceived responsiveness |
A single threshold on the detector's score sets the operating point. Plotting FRR against FA/h across thresholds gives a detection error trade-off (DET) curve, and products are compared at a fixed false-accept budget. False accepts are counted per hour rather than as a percentage because the negative class is unbounded: a device hears thousands of hours of non-wake audio in its life.
False accepts also have a privacy cost. Each one sends a snippet of private conversation to the full recogniser, and possibly to a server.
Choosing a wake phrase#
- Length. Three or four syllables is common. Very short phrases collide with everyday speech.
- Distinct sounds. Uncommon consonant clusters and vowel sequences reduce confusion with ordinary words.
- Rare in conversation. A brand name works better than a common word, which is one reason assistants use invented or unusual names.
- Works across accents. Test with speakers of every language and accent you ship to. A phrase that is easy for native English speakers can be awkward in European English or other languages.
Beyond wake words#
Keyword spotting also appears in in-car systems for a small set of offline commands, in hearing aids and wearables, and in industrial equipment where a spoken "stop" must work without a network. The same trade-off applies: a small fixed vocabulary buys reliability and low power at the cost of flexibility.
Where the vocabulary is open-ended, such as product names, identifiers or medical terms, a fixed detector is the wrong tool. A general recogniser with context biasing or domain adaptation handles thousands of terms without training a detector for each.
Common questions#
Does a smart speaker record everything?
The wake-word model processes audio continuously on the device, in a short rolling buffer. Designs differ on what happens after a detection, including false ones, so check the product's documentation for what is sent and stored.
Can I train my own wake word?
Yes. Commercial toolkits such as Picovoice Porcupine and open-source projects such as openWakeWord support custom phrases, often trained partly on synthetic speech. Budget time for testing false accepts in real environments.
Is keyword spotting the same as searching a transcript?
No. A keyword spotter only knows its fixed phrases. Searching a transcript works for any word, but depends on the recogniser getting that word right.
Why do wake words fail more for some speakers?
Training data under-represents some accents, ages and voices. Evaluate false reject rates separately for each group you serve.
References#
- Chen, G., Parada, C., & Heigold, G. (2014). Small-footprint keyword spotting using deep neural networks. ICASSP.
- Sainath, T. N., & Parada, C. (2015). Convolutional neural networks for small-footprint keyword spotting. Interspeech.
- Zhang, Y., Suda, N., Lai, L., & Chandra, V. (2017). Hello Edge: Keyword Spotting on Microcontrollers. arXiv:1711.07128.
- Warden, P. (2018). Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition. arXiv:1804.03209.
- Rohlicek, J. R., Russell, W., Roukos, S., & Gish, H. (1989). Continuous hidden Markov modeling for speaker-independent word spotting. ICASSP.
- openWakeWord. GitHub.