How we detect interruptions on noisy Indian phone lines

Barge-in sounds simple: when the caller talks, the agent stops. On a real Indian phone call it's one of the hardest parts of a voice agent to get right. The line carries traffic, a pressure cooker, a TV, a second person in the room and the agent's own voice echoing back from a speakerphone. Callers also say "haan", "hmm" and "accha" while listening, which isn't a request to stop.

Our first version stopped the agent on any speech the transcriber reported. It cut itself off constantly. This is what we run now.

Two steps: pause, then decide

The key change was splitting one decision into two.

1. Pause. When we think the caller might be interrupting, we take the agent's audio off the line at once, but keep its reply going underneath. Nothing is thrown away yet.
2. Decide. If the caller's turn ends with real words, we cut the reply and answer them. If nothing real follows within about two seconds, we resume the reply from where it paused, going back a quarter of a second so the caller hears the whole word.

Pausing is cheap and quick to undo. Cutting isn't: the reply is gone and the agent has to start again. So we can pause on weaker evidence and save the cut for strong evidence. A cough costs the caller a moment of silence, not a broken sentence.

What counts as a real word

The pause needs at least a set number of real words (two by default). Getting "real" right took most of the work.

• Fillers and listening sounds don't count: "haan", "haan ji", "hmm", "ok", "theek hai", "जी". These are backchannels. We keep them in the transcript but never answer them mid-reply.
• Punctuation doesn't count, including the Devanagari "।" that some transcribers emit on their own.
• Repeats count once, so "hello hello" is one word.
• Echo doesn't count. On speakerphone, the transcriber often hears the agent's own words coming back. We compare what the caller "said" with what the agent just spoke and drop the overlap.

Stop phrases are the exception. "Wait", "ruko", "ek minute", "band karo" and "nahi" interrupt at once, however short. When someone says "ruko", they mean it.

Stable words, with a voice behind them

Streaming transcribers revise their guesses. A burst of noise can briefly come out as a word, then disappear in the next update. So the words have to hold for two updates in a row before they count.

They also need a voice behind them. We run a voice activity model on the caller's audio and learn how loud the caller's own voice is. Two things are filtered out:

• words with no voice behind them, usually the transcriber's guess at noise;
• voices much quieter than the caller's, typically more than 10 dB down: the TV, or people talking in the background.

Neither is thrown away. Once the agent finishes speaking, they're answered like any other turn, so a quiet caller is never ignored. They just can't stop the agent mid-sentence.

Keeping only what was heard

When a reply is cut, the agent must remember only the part the caller actually heard. Otherwise it later refers to something it never said out loud. That part is harder to find than it sounds. Phone audio is sent to the carrier faster than real time, so by the time the caller interrupts, we may already have sent several seconds that haven't played yet.

We track playback, not sending. The carrier acknowledges checkpoints as audio plays, and between checkpoints we work out when each chunk was due to play. The transcript keeps only the heard part, marked as interrupted, and so does the agent's memory of the conversation.

Things we never cut

Some moments are protected:

• The greeting, by default. The start of a call is full of "hello? hello?" and line noise. What the caller says during it is answered right after.
• A running tool call. Cutting it would lose the request halfway, or send it twice.
• The goodbye. Once the agent has decided to end or transfer the call, the closing line plays in full.

Measuring it

Every call records three counts: real interruptions (the reply was cut), false ones (it paused for noise and carried on) and ignored ones (noise, backchannels or echo that never paused it). We also replay recorded scenarios through real speech-to-text in a test suite: street noise, a TV, echo and fast backchannels. That way a change that fixes one case can't quietly break another.

For noisy environments, a noisy line setting raises the bar: three real words, held for three updates, with a stronger voice, and a longer wait before resuming. Field staff and delivery partners calling from the road need it. Most callers don't.