Voice activity detection, endpoint detection and turn detection are three separate decisions that are often run together. VAD asks whether someone is speaking in this frame. Endpointing asks whether the current utterance is over. Turn detection asks whether the speaker has finished and expects a reply. Each builds on the one before, and each fails in a different way when used in place of another.
Key takeaways
- VAD works on frames of tens of milliseconds and knows nothing about language. Endpointing works on utterances. Turn detection works on conversational intent.
- A pause is silence to a VAD, possibly an endpoint to an endpointer, and often not the end of a turn.
- Using a VAD silence timer as a turn detector is the most common cause of voice agents that interrupt callers or feel slow.
- Barge-in detection is the reverse problem: detecting that a user has started a turn while the agent is speaking. It needs VAD speed with turn-detector judgement.
- The three can be separate components or folded into one model. What matters is that each decision is made with the right evidence.
Three questions, three time scales#
The confusion comes from vocabulary. Vendors and frameworks use "VAD", "endpointing" and "turn detection" loosely, and some products call a VAD with a silence timer a turn detector. The underlying questions are distinct.
| Voice activity detection | Endpoint detection | Turn detection | |
|---|---|---|---|
| Question | Is there speech in this frame? | Is this utterance complete? | Has the speaker yielded the floor? |
| Unit | Frame, 10–32 ms | Utterance, seconds | Conversational turn |
| Evidence | Acoustics only | Silence duration, often decoder output | Words, prosody, dialogue context |
| Output | Speech probability | Utterance boundary, final transcript | Permission to respond |
| Typical failure | Clips words, accepts noise | Splits or merges utterances | Interrupts, or waits too long |
| Used by | Almost every speech pipeline | Streaming ASR, dictation, captions | Voice agents |
Voice activity detection#
VAD is a frame classifier. It is fast, cheap and language-independent, and that is all it should be asked to do. A VAD that correctly labels a 600 ms pause between "my booking to" and "Friday" as silence is working perfectly. The mistake is treating that correct label as a signal that the caller has finished.
Endpoint detection#
Endpointing groups speech into utterances and decides when to finalise a transcript. The basic version is VAD plus a silence timer; better versions add decoder features or emit an end-of-query token from the recognition model. Endpointing was developed for voice search and dictation, where one utterance is usually one request. It does not need to know what the other party in a conversation is doing.
Turn detection#
Turn detection is a conversational decision. Conversation analysts describe turn-taking in terms of transition relevance places, points where a change of speaker is appropriate, following the model set out by Sacks, Schegloff and Jefferson in 1974. Humans predict these points from syntax, intonation and meaning, and start planning a response before the other person stops.
A turn detector tries to do the same. It needs evidence a VAD does not have: whether the words so far form a complete request, whether the pitch fell or stayed level, whether the agent just asked a question with a long expected answer. Open models such as Pipecat's smart-turn use audio of the utterance ending; others use the transcript and a small language model; the strongest combine both.
Barge-in: the reverse problem#
While the agent is speaking, the question flips: has the user started a turn? A VAD fires fast but also fires on backchannels ("mm-hm", "yes") that are not interruptions, on coughs, and on the agent's own voice if echo cancellation is imperfect. A pure turn detector is too slow to stop the agent's speech promptly. Good barge-in handling uses VAD for a fast provisional signal, then confirms with recognised words before cutting the agent off or discarding its reply.
Putting them together#
A typical voice agent pipeline uses all three:
- VAD gates audio and detects onset, including barge-in candidates.
- The streaming recogniser produces partials and endpoints utterances.
- The turn detector decides, at each pause, whether to respond now or keep listening, and adjusts how long to wait.
Frameworks such as Pipecat and LiveKit Agents expose each of these as a configurable component; see voice agent frameworks. The alternative is to fold endpointing and turn prediction into the speech recogniser, which has the words and the acoustics in one place and can emit the final transcript at the moment it predicts the turn is over.
How Reson8 approaches this#
Reson8's Turns endpoint takes the second approach: streaming speech-to-text that predicts end of turn eagerly from acoustic and linguistic signal, instead of a fixed silence timer, with native barge-in detection. On the open Pipecat STT benchmark (1,000 samples, smart-turn-data-v3.1) it reported 326 ms median time from end of speech to final transcript and 1.10% pooled semantic WER.
Common questions#
Can I use a VAD as a turn detector?
You can, with a silence timer, and many early voice agents did. Expect a hard trade-off: a short timer interrupts callers who pause, a long one makes every reply slow. A turn detector exists to escape that trade-off.
Is endpointing the same as end-of-turn detection?
They overlap. In a one-shot voice command the end of the utterance is the end of the turn. In conversation a speaker can pause at the end of a clause and keep the floor, so the two decisions diverge.
Does turn detection depend on language?
Yes. Intonation patterns, sentence-final particles and word order all differ. A model trained on English cues can misjudge completion in Dutch or Polish, so evaluate turn detection per language.
References#
- Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language, 50(4).
- Stivers, T. et al. (2009). Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences, 106(26).
- Skantze, G. (2021). Turn-taking in Conversational Systems and Human-Robot Interaction: A Review. Computer Speech & Language, 67.
- Chang, S.-Y., Prabhavalkar, R., He, Y., Sainath, T. N., & Simko, G. (2019). Joint Endpointing and Decoding with End-to-End Models. ICASSP.
- Pipecat. smart-turn: open-source turn detection model. GitHub.
- Silero VAD. GitHub.