r/VoiceAutomationAI

for the last couple years the voice ai story was one sided businesses automated the receiving end every call center dentist office and airline hotline got some flavor of ai answering the phone but the caller side stayed human because no assistant would actually pick up the phone and talk to someone for you

thats breaking right now multiple companies are converging on the same idea within weeks of each other consumer facing agents that will call a restaurant a clinic a business and have the actual conversation on your behalf combine that with how fast full duplex voice models have gotten sub 300ms response times no more turn detection lag and you get something that sounds completely natural on both ends

which means were heading toward calls where the businesses ai agent answers and the customers ai agent is the one calling neither side is a human being and depending on how well disclosure rules actually get enforced neither side may even announce that clearly the eus already trying to mandate disclosure at the start of every ai interaction but enforcement across phone systems that route through a dozen countries is a very different problem than enforcing it on a website

i dont think this is a bad idea on its face plenty of calls are genuinely tedious and dont need a human on either end but i think people are underestimating how weird its going to feel once its normal and how easy it becomes to lose track of when youre actually talking to a person versus when everyone in the chain is automated

curious where people land on this efficient automation doing exactly what it should or the start of something that quietly erodes what a phone call even is

flagging as i said i would i used ai to help me pull the recent developments together and tighten the writing on this one the take is mine just drafted with help

Demos are usually a few mins long

I’m more interested in what happens 20-30 mins into a call after the customer has changed topics, provided a bunch of information and already completed a few steps.

Does the agent still understand what has happened so far or does context start getting messy?

Anyone testing long voice AI calls in production?

Every voice agent looks good when:

• the caller waits patiently
• the room is silent
• the network is perfect
• the caller answers exactly what was asked
• every backend tool responds in 200 ms

That is not a test. That is a rehearsal.

We had one internal flow that looked almost perfect.

Caller asks to reschedule an appointment. Agent confirms identity, finds the booking, offers a new slot and completes the change.

Then the ugly version:

Caller talks over the greeting.

Pauses for six seconds while checking the date.

Someone in the background says “Friday”.

Caller corrects themselves: “No wait, next Monday.”

Audio drops for a second.

Scheduling API takes three seconds.

Agent starts speaking before the tool result comes back.

Then the transfer fails and the caller gets dead air.

The transcript still looks surprisingly fine.

That is the problem.

Transcript correctness does not tell you:

• whether endpointing cut the caller off
• how long the first response took
• whether the agent stopped during barge-in
• how much dead air happened
• whether it confirmed before the backend succeeded
• whether the transfer actually connected
• whether the caller had to repeat themselves three times

I’ve been looking at TestMu Agent Testing for this because it tests the whole voice/phone flow rather than only scoring the final text.

You can run different personas, accents, noise conditions, interruptions and tool-call scenarios, then compare the behaviour after a prompt or model change.

It can also analyse production recordings, which matters because real callers invent failures no test designer thinks of.

Cekura is strong on newer voice-agent QA and production monitoring.

Cyara and Empirix make a lot of sense for enterprise contact-centre and telephony-heavy environments.

TestMu’s advantage is the broader end-to-end surface across voice, inbound/outbound phone, chat and other agent types.

But “supports 50+ accents” does not mean every language and accent will be evaluated equally well.

You still need to benchmark against your own callers, providers and traffic.

A scripted demo should prove the concept works.

It should not be treated as evidence that the agent is production-ready.

Which real call condition destroys your voice agent fastest?

Push too many calls too fast on a brand new number and carriers flag it as spam. Then the number is dead and the client's campaign is stuck.

So I'm trying to figure out the warm-up part.

If you've run real volume on your own numbers:

- How many calls do you make on day 1 with a new number?

- How long before you're at full volume?

- What daily limit do you stick to per number?

Also curious what actually gets a number flagged. Is it the number of calls, or is it more about people hanging up fast and not answering?

We're on SIP trunking, mostly Indian numbers with some international. Would rather learn this from someone who has already burned a few numbers than find out mid campaign.

Happy to share what we see on our side once we have real data.

How can I build shared context between WhatsApp and an AI voice calling agent?

I'm building an AI system where a customer can communicate with the same AI through WhatsApp and voice calls.

For example:

1. A customer starts chatting with the AI on WhatsApp.

2. During the conversation, they ask for a phone call.

3. The AI voice agent calls them.

4. The voice agent should already know the relevant WhatsApp conversation and continue from the same context instead of starting from scratch.

5. After the call, the customer returns to WhatsApp.

6. The WhatsApp AI should know what was discussed during the call and continue from that point.

And the reverse should also work:

Voice call → WhatsApp → same context

I want the customer to feel like they're talking to one AI, regardless of the channel.

I'm considering using a central customer ID linked to the phone number and storing the conversation history/customer information in a database, so both the WhatsApp agent and voice agent can access the same context.

However, I'm unsure about the best architecture.

- What is the best way to maintain shared context between WhatsApp and a voice AI agent?

- Should I use a central database/memory layer?

- How should I identify the same customer across both channels?

- How should the WhatsApp → voice context handoff work?

- How should the voice → WhatsApp context handoff work?

- How can I prevent the AI from getting confused by multiple summaries or different conversation contexts?

- Has anyone built something similar using WhatsApp Business API, n8n, GHL, or another CRM?

I'm looking for a practical, production-ready approach rather than just passing the entire previous transcript to the AI every time.

I have a bot on the Cloud API that handles text. Message comes in, an LLM answers it with a few tools attached, reply goes back out. That part is stable. Now I want people to be able to call the same number and talk to it, and I am stuck on how to wire the audio.

Two shapes I can see. First is a pipeline. Take the call audio, run STT, hand the transcript to the same model that already answers the text messages, TTS the reply and stream it back. The brain already exists so I would mostly be wrapping it, but that is three hops of latency and I would be writing my own turn detection and barge in handling.

Second is putting a speech to speech model directly in the call. Grok's Voice Agent API does this, OpenAI Realtime does too, audio in and audio out with function calling built in. Much less of my code sits in the path that way. Cost is that my prompt and tools need porting, and I lose the transcript step where basically all my debugging happens today.

From the docs the Calling API gives WebRTC media by default with SIP available if you enable it, so reaching the stream looks fine on paper. Less clear to me is the 2000 conversations per 24h messaging limit before production calling unlocks. Sandbox numbers get around that for testing, which only proves the plumbing works.

If you have run either version with actual users on it, how bad is the latency in practice, and how often does the model end up talking over the caller. Mine is an India number if that changes anything.

been building voice AI pipelines for about two years and i need to say something the hype cycle is burying right now

elevenlabs has genuinely the best voice quality in the space. not close. 11,000 voices, 70 plus languages, sub 100ms latency on voice generation, the IBM watsonx partnership for enterprise. the february raise at $11B was obviously massive and the brand recognition is real. but here is the thing that keeps coming up in every honest thread i've seen recently

you can prototype an elevenlabs voice agent in fifteen minutes. getting it into production as an actual phone agent that handles real customer calls is a completely different story. telephony still requires you to set up twilio or vonage yourself. production monitoring is thin by the platform's own design. HIPAA is locked behind enterprise tier pricing. the reasoning LLM and telephony are billed separately on top of the plan

so you're paying elevenlabs prices for voice quality and then stitching together the rest of the stack yourself...

vapi gives you the full orchestration layer, 14 plus provider connections, 62 million monthly calls processed, 99.99 percent SLA. retell ships a working production agent the same afternoon and leads on turn-taking quality for fast conversational flow. both handle the actual telephony problem that elevenlabs pushes back to you...

the frustrating thing is elevenlabs voice quality is so good that every other platform integrates it anyway. retell uses elevenlabs voices. vapi lets you plug in elevenlabs TTS. so you can get the voice quality without choosing elevenlabs as your agent platform

my actual take: elevenlabs is the best voice layer in the market and the worst standalone agent platform for production use cases right now. the $11B valuation is pricing in what the product will be in two years not what it actually does today

hi guys have a client in Indonesia and Philippines who loved our English voice AI demo, but now wants one in Bahasa and Philippine English/Taglish mixed language, code switching and all

anyone here have real experience deploying voice agents in SEA, specifically TTS that handles code switching well?

what's actually held up in production what sounds good in a demo

We are a startup and have built our AI Voice Agent stack. It runs decently and after painstaking efforts with our mule partner we were able to narrow down the architecture and design to curb failure points as much as we could. However, the biggest pain point we have stumbled across is the telephony carrier!

There is no reliable one! Here is what we have tried and encountered so far:

1. Vobiz: Our current provider. Easy enough to authenticate using personal Adhaar and PAN. They have one of the easiest integrations, setup, and starting curve. the plan pricing is optimum to get started and they don't setup minimum deposit walls. Their API is fantastic to the point that it supports almost all the features that you would need. That being said, there have been issues that we have been facing: The call quality and call handling has been giving us some issues intermittently, where the end result is silent calls. There are other issues including mid call disconnect/call-silence, which I hope to resolve with Vobiz support. Will update if we are or aren't able to resolve this with Vobiz.
2. VoiceLink: Again easy starting with adhaar and pan verification. Decent setup and API support but has a steep starting paywall. They need a minimum of 5000 Rs to get you started without even allowing you to test if their service is compatible and a good fit for your stack. API is good but not great. For example, during our tests Call Transfers would fail there was no way to trace/figure out what happened?
3. Exotel: Hardest to start so far. Requires proper company documentation. Good free tier. Inconsistent pricing information. Hidden credit consumption, cost, and plan information. High paywall with a minimum of 10000 Rs to get started. Support has been great so far where you are actually able to connect with a human who can answer your questions in contrast to the above 2. Credits are timebound for 7 days. After that the free tier ends. API has been good so far, but we are still evaluation Exotel.
4. Plivo: The most scummy of them so far. We couldn't even get an account. They force/tried to sell us their $1000 USD per month plan to get started.
5. Twilio: The most easiest to get started with it ticking green in all the check boxes. Unfortunately, they are not functional in India.

Anyone has any other carrier that they have been working with and can recommend?

Is there a way to make the voice AI model talk back to you normally like it does not pause or something, just like how you talk live?

Every voice agent looks good when:
- the caller waits patiently
- the room is silent
- the network is perfect
- the caller answers exactly what was asked
- every backend tools reponds in 200 ms

That is not a test. That is a rehearsal.

We had one internal flow that looked almost perfect.

Caller asks to reschedule an appointment. Agent confirms identity, finds the booking, offers a new slot and completes the change.

Then the ugly version:

Caller talks over the greeting.

Pauses for six seconds while checking the date.

Someone in the background says “Friday”.

Caller corrects themselves: “No wait, next Monday”.

Audio drops for a second.

Scheduling API takes three seconds.

Agent starts speaking before the tool result comes back.

Then the transfer fails and the caller gets dead air.

The transcript still looks suprisingly fine.

That is the problem.

Transcript correctness does not tell you:
- whether endpointing cut the caller off
- how long the first response took
- whether the agent stopped during the barge-in
- how much dead air happened
- whether it confirmed before the backend succeeded
- whether the transfer actually connected
- whether the caller had to repeat themselves three times

I’ve been looking at TestMu Agent testing for this because it tests the whole voice/phone flow rather than only scoring the final text.

You can run different personas, accents, noise conditions, interruptions and tool-call scenarios, then compare the the behaviour after a prompt or model change.

It can also analyse production recordings, which matters because real callers invent failures no test designer thinks of.

Cekura is strong on newer-voice-agent QA and production monitoring.

Cyara and Empirix make a lot of sense for enterprise contact-centre and telephony-heavy environments.

TestMu’s advantage is the broader end-to-end surface accross voice, inbound/outbound phone, chat, and other agency types.

But “supports 50+ accents” does not mean every language and accent will be evaluated equally well.

You still need to to benchmark against your own callers, providers and traffic.

A scripted demo should prove the concept works.

It should not be treated as evidence that the agent is production-ready.

Which real call condition destroys your voice agent fastest?

Im building a voice agent that can accommodate people from countries like israel, UAE and somewhere around those areas. im struggling to find model that sounds natural and human in those type of languages.

currently using vapi built in voice model which is the elliot since it's expressive but it's american and when changed to different language the american accent is heavily noticable and sometimes goes way off on the guardrails that it speaks gibberish

Note: im new to this niche, i would appreciate some tips to improve thank you!!

voice implementations rn generally fall into two buckets:

1. laggy and robotic api wrappers
2. speech models that are fast, but lack memory and state controls

by building a cascaded stack (deepgram nova-3 → claude haiku 4.5 → elevenlabs flash v2.5), you can keep full control over tool calls and memory, allowing latency reduction. some techniques ive used in my side projects:

• pre-warm anthropic's ephemeral prompt cache while the phone rings
• persistent websocket handshakes and http/2 pool priming on ring
• neural turn-detection with false-interruption resumption (a cough won't kill the tts buffer)
• dual-store memory (sql facts + temporal graph) mapped into a ~300-token prompt snapshot
• proactive outbound scheduling that wakes a killed ios app via apns voip push -> callkit

synthetic ci gates hit p50 ≈ 973ms, though live networks push us to ~3.7s right now (stt and tts ttfb are the real boss fights). Judge our results yourself at getfriendo.app/launch

Hi Everyone,

I am an iOS developer specializing in building complex, offline-first architectures, deep audio routing, and on-device machine learning. If your startup or enterprise needs to process sensitive data directly on the device without relying on expensive (or privacy-violating) cloud APIs, I can help.

Most recently, I architected and built an **Offline Edge AI Voice Logger** from scratch for high-noise industrial environments.

**Key features of my recent architecture include:**

* **Deep Audio Routing:** Built a custom `AVAudioEngine` pipeline with aggressive equalization nodes to filter out heavy background/machinery noise.
* **100% Offline Transcription:** Implemented `SFSpeechRecognizer` forcing on-device recognition, ensuring zero data leaves the iPad/iPhone.
* **Edge Compute NLP:** Trained and integrated a custom `Core ML` text-classification model that parses raw speech into structured, categorized data.

**What I can build for you:**

* Privacy-first iOS applications using on-device Core ML models.
* Complex audio/voice applications (podcasting, dictation, or accessibility tools) utilizing AVFoundation.
* Hands-free / Kiosk applications for medical, retail, or industrial settings.

If your project requires this level of architectural ownership and native framework expertise, please send me a Reddit DM or reach out to me at `gokulayyappath@gmail.com`.