Switching voice provider in minutes
I made a mistake that ended up costing me a couple of months, so maybe this saves someone else the same pain.
My first voice agent stack was built with Retell AI. The backend was n8n, client data lived in PocketBase, and I had a custom CRM built in Next.js. Everything was hosted on Google Cloud.
The setup worked, but hosting costs kept increasing, so I migrated from PocketBase to Supabase. The built-in Row Level Security (RLS) was a huge upgrade.
Later, I wanted to move away from Retell because of the pricing, so I rebuilt everything using LiveKit.
That's where I made the biggest mistake.
I connected LiveKit directly to my CRM instead of putting my backend in the middle. It looked cleaner at first, but when something broke, I had no visibility into what was happening. Every bug meant digging through application logs and tracing code manually.
With my earlier n8n backend, I always knew exactly where a request failed. Without that middle layer, debugging became painful.
After spending weeks trying to make it work, I dropped the project.
Then I tried Dograh instead of LiveKit.
Unfortunately, I repeated the exact same mistake. I connected Dograh directly to the CRM again, without a proper backend layer. Same problems. Same debugging nightmare.
That was the point where I stopped chasing providers and redesigned the architecture instead.
Now I have a custom CRM backed by a dedicated n8n backend running on my own server.
Whether I use Retell, LiveKit, Dograh, or any future voice platform, I just create the agent, connect it over WebSocket or API, and everything plugs into the same backend.
The CRM never needs to know which provider I'm using. Swapping providers now takes minutes instead of weeks.The biggest lesson wasn't about Retell, LiveKit, or Dograh.
It was this:
Never let your frontend depend directly on your voice provider. Put your own backend in the middle. You gain observability, flexibility, and the freedom to switch providers whenever you want.
One thing I genuinely believe, though: self-hosting the open-source version of LiveKit or Dograh on your own server gives you much more control over the entire system. The infrastructure cost is relatively low, but building and maintaining that stack properly is still a significant engineering effort. Realistically, it would take me another 3-4 months to get it to the level I'd want.
Right now, this architecture gives me the best balance between speed, flexibility, and maintainability. I can switch providers in minutes instead of rebuilding my entire system every time.