How does voice activity detection work?
A VAD splits audio into short frames, typically tens of milliseconds long, and labels each one as speech or not speech. Simple detectors compare each frame’s energy with a threshold. Better ones also look at the shape of the sound’s spectrum, and many current ones are small neural networks trained to tell speech from noise, music and clicks.
The threshold sets the trade-off. Set it low and coughs, keyboards and televisions count as speech; set it high and quiet speakers are missed.
Telephony made VAD standard. ITU-T G.729 Annex B, from 1996, is a silence compression scheme built on voice activity detection and discontinuous transmission: during silence, the sender stops sending speech, and the IETF’s RFC 3389 carries comfort noise parameters so the receiver can play background noise instead of dead air.
In voice AI, VAD marks speech turns. OpenAI’s Realtime API, for example, can send input_audio_buffer.speech_started and input_audio_buffer.speech_stopped events, with settings for the activation threshold, the audio kept from before speech began and the silence that ends a turn. Its example uses a threshold of 0.5, 300 milliseconds of padding and 500 milliseconds of silence.
Why does VAD matter for a voice agent?
Turn-taking depends on it. If the detector fires on background noise, the agent may stop mid-answer or reply to a sound, and if it misses soft speech, the caller feels ignored. The same detection decides when talking over the agent counts as an interruption, which is called barge-in.
Spotting the end of speech is the harder half, because a pause to think looks like silence. Many systems pair VAD with endpointing rules or a model that judges whether the sentence is finished, such as OpenAI’s semantic VAD.
Echo matters too. Without echo cancellation, the detector can hear the agent’s own voice through the caller’s speakers and treat it as the caller.
How does Kawl Now handle voice activity detection?
Kawl Now doesn’t run its own voice activity detection, and there are no VAD settings to tune. Calls run on OpenAI’s GPT-Live, whose documented session settings cover the model, instructions, history, voice, backend and storage but not turn detection: the model manages turn-taking itself.
What Kawl Now controls is the microphone and the agent’s instructions. The widget asks the browser for echo cancellation and noise suppression, the mute button stops the caller’s audio, and the agent is told to stop speaking and listen when the caller interrupts.
Interruption handling hasn’t yet been verified in a recorded test with a person speaking into a browser microphone. Before you switch on customer calls, test in a noisy room, pause mid-sentence and talk over the agent.
Common questions
What is the difference between VAD and endpointing?
VAD labels each moment of audio as speech or not speech. Endpointing uses that, and sometimes the words themselves, to decide that a speaker has finished, for example after a set length of silence.
Does voice activity detection work in noisy places?
Less reliably. Noise, music and other voices can be mistaken for speech, so systems raise the threshold, apply noise suppression or use detectors trained on noisy audio. Testing where your users actually call from is the best check.