r/Asterisk

Spent longer than I want to admit on this one, so posting it in case it saves someone else an afternoon.

We were building an AI voice agent against app_audiosocket. Your text to speech hands you a finished sentence, say 2.4 seconds of slin16 at 8 kHz, which is 120 frames of 320 bytes. The obvious thing is to write all of it to the socket and move on.

Don't. app_audiosocket forwards every frame to the channel the moment it arrives. It doesn't buffer for you. So all 120 frames hit the far end in a few milliseconds, the jitter buffer keeps a handful, and the rest get thrown away. The caller hears the tail of the sentence and nothing before it. Your logs look clean the whole time, which is what makes it annoying to chase.

The fix is to write one 320 byte frame, then wait until the next 20 ms deadline. The part that caught us a second time: re-clamp the deadline on every frame. If synthesis stalls for half a second and you work out the next deadline from where you are right now, the writer bursts to catch up and you're dropping audio again.

Same idea with barge-in. When the caller starts talking you have to drop the frames already queued, not just stop writing, or they keep hearing the old sentence for however deep your buffer runs.

We open sourced what we ended up with. asterisk-ai-voice-agent is the Python sidecar (Whisper or ElevenLabs Scribe in, Claude for the turn, Piper locally or ElevenLabs out, barge-in, tool calls posted to a webhook you own). asterisk-audiosocket is just the protocol layer for Node if you'd rather build your own and skip rediscovering the pacing thing. Both MIT, both v0.1.0, so lab material rather than something to point a live queue at.

https://github.com/ictinnovations/asterisk-ai-voice-agent

https://github.com/ictinnovations/asterisk-audiosocket

Happy to answer anything on the AudioSocket side. Curious whether anyone here went the ARI external media route instead and how the pacing compares.