Independent, open source benchmark of voice agent latency - 499 real phone calls, 1,883 turns, every vendor config published
We placed 500 real phone calls across five voice agent platforms — Telnyx, ElevenLabs, Bland AI, Vapi and Retell AI — and measured time to first audio byte on every turn. 499 calls were usable, four scripted turns each, 1,883 usable turns total. Same caller, same carrier (Plivo), same script for all five.
The p50 ranking and the p95 ranking are not the same ranking
Platform p50 p90 p95 p99 turns
---------------------------------------------
Telnyx 1302 1724 1839 2164 379
ElevenLabs 1430 1686 1772 2269 389
Bland AI 1528 2010 2273 2860 389
Vapi 1562 1854 2014 2677 345
Retell AI 1738 2096 2224 2764 381
Time to first audio byte, milliseconds. Lower is better.
Telnyx has the fastest median at 1302 ms. From p90 onward it's ElevenLabs — 1686 at p90, 1772 at p95. Bland AI is third at the median and last at p95, at 2273 ms.
The tail is where they separate
ElevenLabs 1430 → 2269 +839
Telnyx 1302 → 2164 +862
Retell AI 1738 → 2764 +1026
Vapi 1562 → 2677 +1115
Bland AI 1528 → 2860 +1332
Milliseconds added between p50 and p99. Lower is a flatter tail.
Bland AI adds 1332 ms between its median turn and its 99th-percentile turn. ElevenLabs adds 839. On a four-turn call, the p99 turn is the one the caller remembers.
What the numbers are and are not
Recording-path overhead sits inside every figure on this board. We have not characterised the current measurement path against a known-delay reference, so we quote no overhead figure and subtract none. These numbers are comparable to each other — same path, same caller, same carrier — and only approximately comparable to figures produced by a different instrument. If you're looking for each platform's true absolute latency, this board does not give it to you, and neither does anything else we've seen.
Three caveats
- Each platform ran its own defaults. Telnyx on Kimi-K2.6, ElevenLabs on gemini-2.5-flash, Retell and Vapi on gpt-4.1. So this is platforms as shipped, not a model-controlled comparison — some of the gap is model choice. Every config is published, hashed, so you can see exactly what ran.
- Vapi's discard rate was 12.0% against 0.8–3.3% for the others — 46 turns where our two voice-activity detectors disagreed on where speech ended, so we dropped them. Its n is 345 rather than ~385. We'd rather show you a thinner sample than a guessed one, but it's a real asymmetry and it may flatter or penalise Vapi.
- Turn 4 is consistently faster than turns 1–3 on several platforms, which suggests warm-path effects we haven't isolated. Per-turn splits are in the data.
Why phone calls rather than websockets
Most published voice latency numbers benchmark TTS or STT models in isolation. This measures the whole platform over the PSTN — carrier, telephony stack, endpointing, model, voice — because that's the number a caller actually experiences. It's the less flattering way to measure and the harder one to reproduce, which is why the configs and per-turn data are published.
Reproducing it
- Results: https://openbenchmarks.com/voice-agent-latency
- Method: https://openbenchmarks.com/voice-agent-latency/how-voice-agent-latency-is-measured
Happy to add a platform or rerun with a controlled model if that's the comparison you want. If you've measured any of these yourself and got something different, post your numbers — the method page says exactly what we did, so the disagreement should be locatable.