Barge-in: why being able to interrupt a voice agent changes the conversation

The term was already used for interactive voice response systems. The W3C’s VoiceXML 2.0 Recommendation, published on 16 March 2004, describes it as follows: if the platform supports it, the application author chooses whether the user can interrupt a prompt by voice or with a keypad key. The text notes that this speeds up conversations, but that it is not always desirable: for a warning or a legal notice that must be heard in full, barge-in should be disabled.

Today’s conversational platforms have taken up the concept. In Dialogflow CX, Google’s documentation states that when barge-in is enabled, the user can interrupt the response audio: the agent then stops sending audio and processes the new input. Amazon Lex V2 lets you choose, for each piece of information the bot asks for, whether the user can interrupt it while it is speaking.

Between two people, we interrupt to correct (“no, Tuesday, not Wednesday”), to answer before the question is finished or to signal that we have understood. An agent that carries on to the end of its sentence while the caller is talking forces them to repeat themselves, and feels like talking to a brick wall.

Amazon Lex’s documentation gives a use case: a user who already knows the answer to the question being asked, for example when asked for their card’s security code. The VoiceXML Recommendation highlights another effect: stopping the prompt as early as possible avoids the “stuttering” effect, where users stop mid-sentence and start again because they do not think they have been heard.

VoiceXML distinguishes two types of barge-in. With the “speech” type, the prompt stops as soon as speech or a keypress is detected, whether or not the input matches what is expected. With the “hotword” type, the prompt continues until an expected phrase has been fully recognised, and everything else is ignored.

The trade-off is the same today: react quickly at the risk of stopping for nothing, or wait for meaningful speech at the risk of seeming slow. In OpenAI’s Realtime API, the interrupt_response parameter determines, according to the API reference, whether an ongoing response is automatically interrupted when voice activity detection signals the start of speech. The semantic_vad mode, according to the documentation, makes the model less likely to interrupt the user, and the eagerness parameter sets how eager the model is to interrupt the user.

To be interrupted, the agent must listen while it speaks. But its own voice can come back into the captured signal, for example when the caller is using their phone’s loudspeaker. Without processing, the system risks mistaking its own voice for an interruption.

The W3C Media Capture and Streams specification defines echo cancellation as removing, from the signal recorded by the microphone, the sound that is currently being played. On the telephone network side, the International Telecommunication Union devotes an entire recommendation, G.168, to digital network echo cancellers. Barge-in is only reliable if this step works.

Voice activity detection (VAD) decides when someone is speaking. Deepgram’s documentation acknowledges that loud background noise, such as music or a phone ringing, can trigger VAD and prevent detection of the silence that marks the end of a sentence. It offers an alternative based on the gap between transcribed words, which ignores non-speech sounds such as knocking on a door, a phone ringing or street noise.

OpenAI, for its part, states that a higher activation threshold requires louder audio to trigger the model, which may work better in noisy environments, and that a shorter silence duration detects the end of a turn more quickly. These are settings, not universal solutions, and each has its downside: a threshold that is too high risks missing quiet voices, a silence that is too short risks cutting off someone who is thinking.

Barge-in also has a cost. Google states that, in Dialogflow CX, enabling it increases the billable duration of each turn, since the agent listens while it speaks.

A few tests are enough to get an idea: interrupt the agent mid-sentence to correct a date, call on speakerphone, call with a radio on, slip in a brief acknowledgement while it is speaking, pause for a long time before finishing your sentence. Each time, note whether the agent stops, whether it picks up again correctly and whether it has understood the correction. Alyas answers the phone 24/7 using the information you provide and places a transcript and a summary of each call in your customer area, so you can review these tests. The ten-day free trial, with no bank card required, lets you run these tests on your own scenarios.