r/VoiceAutomationAI

What actually determines whether a voice agent "feels real" on a call (latency breakdown from building one)

Disclosure: I build Talkif (voice AI infra), so take the specifics below as one data point, not a review.

Spent the last while digging into why some voice agents feel natural and others feel like talking to an IVR menu. The model and the TTS voice quality matter way less than people assume. The thing that actually breaks the illusion is response gap: anything over ~1 second of dead air after the caller stops talking, and people start repeating themselves or hanging up.

Where the time actually goes, in our experience:

  • Cold-start on the bot process itself. If your agent spins up fresh per call, that's often 1-2 seconds gone before anything else happens.
  • Round trip through whatever telephony layer you're on (Twilio, generic SIP, etc.) - this adds up fast if you're bouncing between regions.
  • Serializing context (CRM lookups, contact history) into the prompt at call time instead of having it ready before the call connects.
  • No visibility into where time is actually being spent, so you're guessing instead of measuring.

Things that moved the needle for us: pre-warming bot instances instead of cold-starting per call, keeping SIP routing close to the caller's region instead of round-tripping across continents, and streaming call events in real time so you can actually see where latency creeps in instead of finding out from an angry customer.

Ended up around sub-1-second first response most of the time, which seems to be roughly the threshold where callers stop noticing they're talking to a bot.

Curious what others here are seeing - anyone measured where their latency actually goes, or is it mostly a black box until something feels slow?

reddit.com
u/Competitive-Fee7222 — 4 hours ago

Built a self-hosted voice agent to get away from vendor lock-in — turns out that's apparently the #1 complaint in this space? (asking, not selling)

Saw a thread here a while back where someone said the biggest issue with proprietary voice-agent platforms isn't the tech, it's the lock-in. That stuck with me, because it's exactly why I started building my own (Twilio + Deepgram + LLM + TTS, all swappable, your own keys, your own database).

Wasn't planning to post this anywhere yet — wanted to get it further along first — but I keep seeing the same three things repeated across this sub and a few others, and want a gut-check before I go further:

  1. Lock-in — is this actually the top blocker for people, or is it more about cost, latency, or something else entirely once you actually try to ship one of these?
  2. Compliance — TCPA/DNC warnings show up on basically every automation-adjacent thread I've read. Is that real lived pain (people have actually gotten close to a violation) or more theoretical fear that doesn't change what people build?
  3. State/memory — anyone here hit agents re-asking for info the caller already gave? I built a structured state layer instead of relying on the transcript as memory, curious if others have solved this differently.

Just trying to figure out if what I've built is actually solving problems people have, or if I've been building for an imaginary user. What's your actual experience?

reddit.com
u/Agreeable_Ask7187 — 9 hours ago
▲ 14 r/VoiceAutomationAI+8 crossposts

I built an open-source Agent Verifier for Claude Code, Cursor & other Coding Assistants that catches security issues, hallucinated tools, infinite loops and anti-patterns in Agent built using LangChain, LangGraph, and other frameworks. (free, open source, 100% local)

I've been using Claude Code for a few months and noticed AI agents consistently skip the same things: hardcoded secrets, unbounded retry loops, referencing tools that don't exist, and massive system prompts that blow context windows.

So I built Agent Verifier — an AI agent skill that acts as an automated reviewer which does more than just code review (check the repo for details - more to be added soon).

GitHub Repo: https://github.com/aurite-ai/agent-verifier

Note: Drop a ⭐ if you find it useful to get more updates as we add more features to this repo.

----

2 Steps to use it:

You install it once and say "verify agent" on any of your agent folder in claude code to get a structured report:

----

✅ 8 checks passed | ⚠️ 3 warnings | ❌ 2 issues

❌ Hardcoded API key at config.py:12 → Move to environment variable
❌ Hallucinated tool reference: execute_sql → Tool referenced but not defined
⚠️ Unbounded loop at agent/loop.py:45 → Add MAX_ITERATIONS constant

----

Install to your claude code:

npx skills add aurite-ai/agent-verifier -a claude-code

OR install for all coding agents:

npx skills add aurite-ai/agent-verifier --all

----

Happy to answer questions about how the agent-verifier works.

We have both:
- pattern-matched (reliable), and,
- heuristic (best-effort) tiers, and every finding is tagged so you know the confidence level.

----

Please share your feedback and would love contributors to expand the project!

u/Chance-Roll-2408 — 17 hours ago

Our team and i have built a TTS model from scratch. AMA.

Voice builders, you guys are some of the rarest, hardest people to find lol. Or maybe i'm looking in the wrong places.

Regardless, We've spent about 6 months straight building a TTS model and boy was it one hell a challenge.

I wanna take some time off to maybe help others who are building in the space, and even learn how we could do things better.

it's designed specifically for realtime conversations.

reddit.com
u/Dynamicrex — 3 days ago

I built OpenVoice AI: a self-hosted AI communication platform starting with an AI phone agent

Hi everyone,

I’m building OpenVoice AI, an open-source self-hosted AI communication platform.

The project starts with an AI phone-agent stack, but the long-term goal is broader: phone calls, email, calendar, dashboards, automations and multiple AI providers in one self-hosted system.

Current v0.1.0 preview includes:

- Asterisk phone-agent baseline

- local STT/TTS foundations with faster-whisper and Piper

- FastAPI dashboard foundation

- call recording and transcript workflow

- email draft foundations

- NVIDIA / OpenRouter provider configuration

- Proxmox LXC helper-script drafts

Repo:

https://github.com/nikpottbecker/openvoice-ai

Important: this is still preview / early alpha, not a polished production release.

The biggest current challenge is real phone-call speech-to-text quality, especially German telephone audio. I’m looking for feedback from people who run Proxmox, Asterisk, SIP, FRITZ!Box, n8n, local AI or self-hosted automation stacks.

Questions:

  1. Would you self-host something like this?
  2. Which integrations would matter most to you?
  3. What would make the setup trustworthy enough to try?
  4. Which STT provider should I test next for telephone audio?

I’m the maintainer of the project. Feedback, criticism and architecture suggestions are welcome.

u/Routine_Beautiful918 — 3 days ago

those of you running voice agents in prod — what actually happens between editing a prompt and real callers hearing it?

We have two voice agents in production. One does outbound calls, the other basically runs day-to-day admin for a small business — their customers talk to it every day.

Embarrassing confession: every time I touch the prompt, my entire testing process is... I call it 5 times, listen, and if it sounds fine, I ship. That's it. That's the pipeline.

We did properly measure it once — sat down and counted, roughly 9 out of 10 voice requests did the right thing. Felt great for a day. But that was one manual count. No idea what the number is today.

So I'm curious what everyone else is actually doing:

- when you change a prompt, does it hit a staging agent or some test-call suite first, or straight to prod like us?

- how do you find out something broke? for us it's honestly "the client texts me"

- did anyone build real tooling for this, or is it duct tape everywhere?

I read a ton of launch posts but nobody ever writes about week 6. genuinely want to know if everyone's secretly winging it like we are, or if we're just behind.

reddit.com
u/t-stroms — 3 days ago
▲ 4 r/VoiceAutomationAI+1 crossposts

Voice agents, demystified: STT+TTS and 4 demo agents you can talk to in the browser + build yours with RAG and Tools

I added voice to AgentSwarms! You can create voice agents using a few clicks and talk to it in the browser — and you can try 4 demo voice agents right now, no setup, just tap the mic. Here's how it works and why it turned out to be less "new" than I expected.

The surprise building this: a voice agent is basically the chat agent you already know, with a voice on top. Same system prompt, same tools, same RAG, memory, and guardrails. Under the hood it's a simple loop — your mic gets transcribed to text (OpenAI GPT-4o-mini-transcribe), your agent replies exactly like it would in chat, and that reply gets spoken back (OpenAI GPT-4o-mini-TTS). The agent's brain doesn't change at all. You've just added ears and a voice.

Which is the whole point: everything you've already learned building chat agents carries straight over. If your agent can pull an answer from a knowledge base, call a tool, or respect a guardrail in text, it does all of that out loud too — because it's the exact same engine with audio on the two ends, not a separate stripped-down "voice mode."

**What I shipped**

* **New Voice Agent** in the builder: pick a voice (11 of them), a greeting, and your STT/TTS models. That's the whole setup.
* Every spoken reply runs the same pipeline as a chat agent — tools, knowledge base, memory, and guardrails all apply.
* A **Voice Playground**: tap the mic, talk, and hear the reply back, with the transcript on screen so you can read along.

**Talk to it (free, in the browser)** — 4 demos, tap the mic:

* **Aria** — customer support triage
* **Nova** — B2B discovery caller
* **Kai** — Spanish conversation tutor
* **Echo** — daily standup coach

Open one, talk to it, and fork it into your own workspace if you like it.

* Voice Playground → [https://agentswarms.fyi/voice-playground\](https://agentswarms.fyi/voice-playground)
* Build your own (New Voice Agent) → [https://agentswarms.fyi/agents\](https://agentswarms.fyi/agents)
* Docs → [https://agentswarms.fyi/docs/voice\](https://agentswarms.fyi/docs/voice)

*Disclosure: AgentSwarms school of Agentic AI for both no-code people and developers— a learn-by-building platform. The demos are free. Happy to answer anything about the setup in the comments.*

![img](upjeq6kua0bh1)

u/Outside-Risk-8912 — 3 days ago

Building an AI voice agent agency looking to connect with other hungry agency owners to share notes and scale

Hey everyone,
I’m currently in the launch phase of building an AI Voice Agent agency .
I’ve spent the last few weeks setting up my full outbound infrastructure—getting secondary domains ready, setting up ESP warmups, and scraping targeted B2B data. Campaigns are ready to fire.
I’m not here to pitch or sell anything. I just want to connect with other hungry agency owners who are in the trenches right now. Building solo can be a grind, and I’m looking to connect with people who share the same hunger to bounce ideas off each other, talk strategy, and share what's working.
Whether you're focused on the tech side (Vapi, n8n, CRMs) or the sales/acquisition side, I'd love to connect.
Drop a comment below or shoot me a DM
Let’s win together!

reddit.com
u/RegionSavings1687 — 5 days ago

How to outreach for Voice AI Clients in the Real Estate Niche

I sell inbound call agents for afterhours and outbound agents for fast followups. I have finalized my niche to Real Estate and built my whole system around it. What is the best way to get clients in this niche. Obviously outreach is important but what is the best method? Cold Emails? Cold calls? Ads (I am not confident about I have never worked on)?

reddit.com
u/Latter-Pianist-3992 — 5 days ago

Lead generation and cold email for an AI Voice Agent SaaS

Hey everyone,
I recently built an AI Voice Agent SaaS using Vapi. My initial target market is medical clinics (dental, aesthetic, hair transplant, etc.) and beauty salons.
The problem I’m facing is that I’m struggling to find enough quality leads. I’ve tried scraping Google Maps and using lead databases, but the number of qualified leads is much lower than I expected.
My plan is to run cold email campaigns, but before I start sending emails at scale, I want to make sure I’m building the right lead list.
For those of you who have experience with B2B SaaS or agency lead generation:
How do you find high-quality leads consistently?
Which tools or data sources have worked best for you?
Do you rely mainly on Google Maps, Apollo, LinkedIn Sales Navigator, or something else?
How do you validate emails before sending?
What cold email strategy has produced the best reply rates for you?
I’d really appreciate hearing what’s actually working in 2026, especially from people selling to local businesses or clinics.
Thanks!

reddit.com
u/Necessary_Future6054 — 6 days ago

Need help to understand pricing logic

hi - i'm looking for some guidance on the pricing model for your ai voice services. I've seen videos on youtube where agency owners do

  • Monthly Retainer + One-time setup fee
  • Per Min Billing + One-time setup fee
  • Value based (Formula: Missed calls/day × booking rate × avg job value × operating days = monthly recovered revenue, Charge: 10–20% of recovered revenue)

I'm starting out so would appreciate your input on what worked for you, the questions you asked the client to gauge the usage and also; ai voice comes with a combination of tools like vapi/retell + n8n + call dashboards - how much margin do you add to these and price your efforts to achieve the final outcome?

reddit.com
u/Alarming_Outcome_946 — 5 days ago
▲ 9 r/VoiceAutomationAI+1 crossposts

[Partner Wanted] Looking for a US-Based Growth/Sales Partner for an AI Voice Agent Venture (Technical Co-Founder inside)

​Hey everyone,

​I'm a technical founder currently building conversational AI voice agents (handling inbound/outbound workflows, CRM syncs, etc.). The technical architecture is fully functional, but I’m looking to connect with someone based in the US to handle the business development and operations side.

​Since I am focused entirely on the engineering, I'm looking for a partner who can take ownership of:

​Go-to-Market Strategy: Identifying the right industries and use cases.

​Outreach & Sales: Managing early-stage discovery calls and client relationships.

​Operations: Navigating the US market and compliance landscapes.

​If you have experience in B2B sales, agency growth, or tech operations and want to team up on a serious voice AI venture, I’d love to chat.

​Drop a comment or send over a DM with a bit about your background!

reddit.com
u/amit5757 — 6 days ago
▲ 3 r/VoiceAutomationAI+2 crossposts

AI Voice calling agent

Hi! We’re looking to collaborate with someone who has already built a production ready AI voice calling agent. We’re not looking for someone to develop one from scratch..we’re specifically seeking an existing solution.

This is a strategic collaboration opportunity focused on educational institutions, with the potential to unlock a large new market together.

reddit.com
u/nexin_solutions — 6 days ago

Spent 6 months building an AI restaurant voice agent. Rejected by Clover, Toast & Deliverect. Would you pivot?

Hi everyone,

I'm looking for some honest advice from founders and engineers who've built products that depend on third-party platforms.

Over the last 6 months, my co-founder and I built an AI phone agent for restaurants. The product is production-ready and can:

• Answer incoming restaurant phone calls
• Take orders conversationally
• Answer menu questions
• Upsell items
• Send SMS order confirmations
• Transfer calls to restaurant staff when needed
• Create orders directly in the POS

The entire AI voice infrastructure is complete and working.

The problem is distribution.

We first integrated with Clover using their official APIs. After months of development and going through their review process, Clover's legal team rejected our app, saying they currently don't have an established policy governing third-party AI applications.

During the review we clarified that:

• Call recording is completely optional and controlled by the restaurant.
• Recordings remain in Twilio and are not stored on our servers.
• We only store transcripts for restaurant review.
• We do not use customer conversations to train foundation AI models.

Even after explaining all of this, the application was still rejected.

We also explored Toast and Deliverect, but those paths haven't worked out either.

What makes this confusing is that companies like Loman AI and Certus AI publicly advertise Clover integrations and appear to provide very similar AI voice ordering functionality.

At this point we're wondering if we're chasing the wrong problem.

Our options seem to be:

  1. Continue integrating with another POS (Square, etc.), knowing it could take several more months with no guarantee of approval.
  2. Remove the POS dependency and become an AI receptionist that sends orders to restaurant staff for manual entry.
  3. Pivot the technology into another industry (HVAC, dental, plumbing, legal, etc.) where phone automation is valuable but doesn't depend on POS approvals.
  4. Something else that we haven't considered.

My questions are:

• Has anyone here built software that required approval from Clover, Toast, Square, Shopify, Salesforce, or another platform?
• Is this kind of rejection just part of building on someone else's ecosystem?
• If you were in our position, would you keep pursuing restaurant POS integrations or pivot before investing another 6+ months?
• Are there other approaches to restaurant automation that don't depend on getting platform approval?

We're not trying to complain about Clover or any specific company. We understand they have legal and security responsibilities. We're simply trying to decide whether this is a normal hurdle that startups eventually overcome, or a signal that our business is too dependent on approvals from platforms we don't control.

I'd really appreciate hearing from anyone who's been through something similar. Thanks!

reddit.com
u/Capital_Act8480 — 5 days ago
▲ 2 r/VoiceAutomationAI+1 crossposts

Building an AI voice agent agency looking to connect with other hungry agency owners to share notes and scale

Hey everyone,
I’m currently in the launch phase of building an AI Voice Agent agency .
I’ve spent the last few weeks setting up my full outbound infrastructure—getting secondary domains ready, setting up ESP warmups, and scraping targeted B2B data. Campaigns are ready to fire.
I’m not here to pitch or sell anything. I just want to connect with other hungry agency owners who are in the trenches right now. Building solo can be a grind, and I’m looking to connect with people who share the same hunger to bounce ideas off each other, talk strategy, and share what's working.
Whether you're focused on the tech side (Vapi, n8n, CRMs) or the sales/acquisition side, I'd love to connect.
Drop a comment below or shoot me a DM
Let’s win together!

reddit.com
u/RegionSavings1687 — 6 days ago

I built an AI phone agent you can actually call right now pick from 10 weird personas (call is recorded for quality and AI training purposes)

🔴🔴🔴This is a recorded line. Every call is recorded and transcribed (I use the recordings to improve the system, and you'll also hear this notice when you call in). By calling, you're consenting to being recorded if you're not cool with that, don't call. Don't share anything private, sensitive, or personally identifying treat it like a public demo, because it is.🔴🔴🔴

🔴🔴🔴It's a regular Canadian number (Quebec, area code 450), not toll-free, calling may cost you long-distance or international rates depending on your carrier and country.🔴🔴🔴

I've been building a self-hosted AI voice agent platform (Asterisk + local GPU models, running on a little 4×RTX 3090 rig in my place). So come call it.

📞 Number: 1-450-400-1274

When you call, you'll get a menu.

Press a key to talk to one of the personas:

  • 0 – Me (the actual human who built this, so, maybe dont?)
  • 1 – Larry's Lambos 🏎️
  • 2 – The Turtle Specialist 🐢
  • 3 – Kim from Hensley and Sons Hardware Supply
  • 4 – Hot Dog Barn 🌭
  • 5 – Gerald from Hensley and Sons Hardware Supply
  • 6 – Edna
  • 7 - Agatha the Astrologist 🔮
  • 8 - Sofia, the Mallorca travel specialist 🏝️
  • 9 – The Chashu Hotline 🍜

Just talk to them like a normal call it's real-time speech-to-speech, no app, no signup.

A few honest caveats:

It's a tiny home rig that can only handle a handful of calls at once. If you get a busy signal or a rejection, it's swamped wait a few minutes and try again.

The AI will say weird/wrong things. That's half the fun. It can't do anything except talk (no transfers to real services, no texting you, nothing).

It's a hobby project, not a company. No data is sold; recordings just live on my box.

Would love feedback on latency, voice quality, and how the personas hold up. Roast away.

🔴🔴🔴This is a recorded line. Every call is recorded and transcribed (I use the recordings to improve the system, and you'll also hear this notice when you call in). By calling, you're consenting to being recorded if you're not cool with that, don't call. Don't share anything private, sensitive, or personally identifying treat it like a public demo, because it is.🔴🔴🔴

🔴🔴🔴It's a regular Canadian number (Quebec, area code 450), not toll-free, calling may cost you long-distance or international rates depending on your carrier and country.🔴🔴🔴

reddit.com
u/LaysWellWithOthers — 8 days ago
▲ 7 r/VoiceAutomationAI+3 crossposts

Quick context: I use Claude Code and Codex daily and noticed I was spending half my "agent is working" time just sitting there watching the screen. I was like, what if Claude or Codex can just narrate its process back to me, so I know what it's doing?

So I built Heard. Open-source.

What it does:

Speaks your agent's intermediate output - tool calls, status updates, the prose between actions. You can get up, make coffee, and still hear when it hits a failure or needs input.

Stack:

- Python daemon, Unix socket, fire-and-forget hooks (never blocks the agent)

- ElevenLabs for cloud TTS, Kokoro for fully local (no key needed)

- Optional Claude Haiku 4.5 for in-character persona rewrites

- Adapters for Claude Code + Codex; `heard run` wraps anything else

- macOS app + CLI, Apache 2.0

What I learned building it:

The hard part wasn't TTS, it was deciding what NOT to say. First version narrated everything and was unbearable in 90 seconds. Now there are 4 verbosity profiles and "swarm mode" for when 2+ agents are running concurrently - background ones only pierce on failures so you don't get audio soup.

Roadmap: Cursor + Aider adapters, Linux/Windows after that.

Would love feedback on features that broke or stuff that you would like to see!

Repo: https://github.com/heardlabs/heard

Voice samples: https://heard.dev

u/decentralizedbee — 9 days ago

Building and selling a 30-hour niche dataset of voice agent workflows (ElevenLabs)

Hey,

I’ve got an idea that on paper seems like an easy way to make a decent amount of money, but I’m not sure how realistic it is.

My plan is to sell datasets of myself building voice agents on ElevenLabs. The dataset would include screen recordings, mouse movements, tab switches, and full annotations.

The technical barrier to capture the screen and interaction data is quite low, and I plan to outsource the video annotation process to a team in Madagascar.

Having already built live voice agents, I know that building them involves a lot of live iteration and quick fixes, which makes this specific sequential data highly valuable for "computer use" AI models.

My strategy: I want to create a 2-hour high-quality sample to showcase on X and Reddit, and use it to cold-pitch potential buyers via email and LinkedIn DMs. If there's proof of interest, I can easily produce a 30-hour dataset within a week or two. Even though it’s not thousands of hours, I believe this specific type of workflow data is rare enough that AI labs might buy it.

My three main questions are:

  1. Is this kind of niche workflow dataset interesting enough for labs or developers to actually buy?
  2. What's the best way to get this in front of the people who have the budget for it?
  3. Is a 30-hour high-quality dataset for a complex task substantial enough to be sold standalone?

I got this idea after seeing Markov in the current YC batch. The data samples on their website look high-quality but totally feasible to replicate for specific niches.

If anyone has thoughts or experience in the AI training data market, I'd love to hear them!

reddit.com
u/Wrong-You-726 — 9 days ago