u/Working_Hat5120

Showed (in the clip) the underlying signals onto the screen instead of hiding them in logs, so that they become measurable.

Entities (which have high confidence) also get captured in-stream before the text hits the action model (an SLM), SVMs.

Observation from this clip:

- Tone it hears, as a distribution, never a single verdict.
- What it thinks you're actually asking for
- And the tool it already ran, before the sentence finished
- That last one is interesting: It doesn't wait for you to stop speaking to start working.

I have few questions:

- Has anyone seen a semantic endpointer beat a tuned VAD in prod?

- Suggestions for workflows I should benchmark, to show effectiveness of my approach.

Happy to go into any of it.

https://reddit.com/link/1vqf904/video/1u9v6auhxujh1/player

Fair warning, I'm giving you the whole pitch so you can roast the actual thing and not my website. My website is fine. It's not the point.

The product
Whissle (whissle.ai). A platform for multi-modal voiceAI agents. The bet: every voice agent today decides from a transcript, and the second speech becomes text we lose how it was said.

the hesitation before "yes," the stress, whether it's even the same person on the line. We built our own speech model that reads words + intent + emotion + speaker + entities in one pass, as live distributions, and a small model (Meta-SLM) that decides the turn on that signal: wait, answer, verify, or escalate.

It speaks back with the emotion from the same pass. We call it Multi-Modal Synchronous Intelligence, which I know sounds like a buzzword generator had a stroke, but it just means: hear, decide, and speak inside the same turn.

Who wants it: Anyone running high call volume where mishearing a name or amount, or missing "I can't actually pay this month," costs money. Government and skill accessment, healthcare, financial services, legal, and live-event / media are where it's landing. Plus regulated shops that can't let raw audio or PII leave their network.

The market
Voice AI is hot and very crowded. Contact-center + voice-agent tooling is a multi-billion category. Our serviceable wedge (real-time, signal-aware, self-hostable voice) we put around $800M-1B, targeting ~$22M ARR by year 3. Yes, it's a hockey stick. Roast accordingly.

Comparison vs the field
Most players stitch someone else's STT to an LLM to someone else's TTS. We benchmark ourselves against 11Labs, Hume, Vapi, and Sierra, and the difference is what we own:

• A 23-language audio (and visual) smart transcription with metadata, not a wrapper on someone's API
• Affect + intent per turn as distributions every interim, not post-call analytics
• PII captured and redacted in-stream, raw identifiers never hit the model or the logs
• A replayable, auditable evidence trail for why each action fired
• Runs hosted, or self-hosted on your own hardware Where we lose today, since you'll find it anyway: on clean read-aloud benchmarks the big cloud APIs still beat us on raw word error rate. We win on real-world noisy and code-switched calls, and on getting the action right, not just the sentence.

Stage / raising
Live now, opening the platform. Traction: 200K+ model and dataset downloads (17 models, 38 datasets open), 15+ enterprise LOIs, pilots running, 5 research papers, backed by NVIDIA Inception, Google Cloud partner, and Lambda research. Raised $200k+ from 15 angels, now raising $1M on a SAFE (20% discount) for 12 months of team and ops expansion. Pricing: $0.06/min all-in for voice agents (US calling included), $20/user/mo for the companion, and private deploy as a license plus a share of the savings.

Customer conversion
Founder-led. Inbound comes from our open models on HuggingFace (devs find us, 200K+ downloads is the top of the funnel), linkedin posts. Warm connects from angel investors. Outbound to healthcare, financial services, and government/skilling targets. We land with a pilot on their single worst call type, the one where a misheard number costs money, prove entity-level accuracy and task completion, then expand. Devs self-serve on the platform.

Why me
Karan Singla, founder/CEO. PhD from USC (SAIL, multi-modal AI), ex-Interactions where I shipped voice AI in production, specially in-stream PII redaction and emotion understanding. Advisors: Shrikanth Narayanan (USC SAIL), Jay Wilpon (ex AT&T / Bell Labs), Antonio Moreno (SoundHound). We publish AI research and people actually run our models, some are building new startups with it. No rich daddy to pull the plug, so this has to work.

What I actually want torn apart:

• Is "the transcript is lossy" a real wedge, or a nerd feature nobody pays for?
• Owning our own speech model in a world of dirt-cheap APIs.. moat, or masochism?
• $0.06/min all-in against metered stacks.. smart, or a speedrun to zero margin?

Have at it. whissle.ai if you want to poke the actual thing. Not a great scaled hosting yet.

Hi all,

We're getting ready to open up our agentic VoiceAI platform (whissle.ai) and want honest feedback from people who actually run receptionists and phone agents before we widen access.

What we did differently: most agents decide from the transcript, so the moment speech becomes text they lose how it was said. Ours reads the words plus intent, emotion and who's speaking in the same pass, so it can act on that mid-call: soften when someone's anxious, verify when the speaker doesn't match, escalate to a human when confidence is low or flipped emotion (rules can be made by agent operator) It also captures and redacts PII (card numbers, DOB) in-stream, so raw identifiers never hit the logs or the downstream model.

What's there today:

• Real-time booking and calendar checks
• Missed-call follow-up (SMS or callback with an owner, not a dropped task)
• Escalation to a human when it's unsure, not just on keywords.
• Metadata understanding for 23 languages, and smart deterministic gating.
• Hosted, or self-hosted on your own hardware so audio stays in your boundary. We monitor and harness
• Replayable decision trail (see why each action fired) for compliance

What I'd genuinely like feedback on:

• For a receptionist, is tone/intent actually useful, or is booking accuracy all that matters?
• What do current platforms get wrong for you.. latency, missed transfers, bad readback of numbers? PII leakage.
• Does data staying on your own infra matter to your clients, or is hosted fine?
• Agencies: what would you need to manage this across multiple clients?

Happy to hand out extended free credits to anyone here who wants to actually test it and tell me what's missing.

Thanks

The moment I transcribe to text, I generally lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone.

For a human those signals, whether to trust the commitment, reconfirm from the caller or escalate to human come naturally but hard to define a deterministic paralinguistic to build accountability, which is probably very wide.

How are you modeling tone in our voice-agents? I see recent TTS models which accept meaningful tags producing great sounding speech, how do we control it ? Does it account for input user's tone.

How does your ASR model / voice-agents captures the tone or there are some good services / models / tools / solutions to capture tone. and how do you use it downstream ?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency. Is there a good duplex model which provides transparency in underlying signals beyond just text.

The moment I transcribe to text, I generally lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone.

For a human those signals, whether to trust the commitment, reconfirm from the caller or escalate to human come naturally but hard to define a deterministic paralinguistic to build accountability, which is probably very wide.

How are you modeling tone in our voice-agents? I see recent TTS models which accept meaningful tags producing great sounding speech, how do we control it ? Does it account for input user's tone.

How does your ASR model / voice-agents captures the tone or there are some good services / models / tools / solutions to capture tone. and how do you use it downstream ?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency. Is there a good duplex model which provides transparency in underlying signals beyond just text.

The moment I transcribe to text, I generally lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone.

For a human those signals, whether to trust the commitment, reconfirm from the caller or escalate to human come naturally but hard to define a deterministic paralinguistic to build accountability, which is probably very wide.

How are you modeling tone in our voice-agents? I see recent TTS models which accept meaningful tags producing great sounding speech, how do we control it ? Does it account for input user's tone.

How does your ASR model / voice-agents captures the tone or there are some good services / models / tools / solutions to capture tone. and how do you use it downstream ?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency. Is there a good duplex model which provides transparency in underlying signals beyond just text.

The moment I transcribe to text, I generally lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone.

For a human those signals, whether to trust the commitment, reconfirm from the caller or escalate to human come naturally but hard to define a deterministic paralinguistic to build accountability, which is probably very wide.

How are you modeling tone in our voice-agents? I see recent TTS models which accept meaningful tags producing great sounding speech, how do we control it ? Does it account for input user's tone.

How does your ASR model / voice-agents captures the tone or there are some good services / models / tools / solutions to capture tone. and how do you use it downstream ?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency. Is there a good duplex model which provides transparency in underlying signals beyond just text.

The moment I transcribe to text, I generally lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone.

For a human those signals, whether to trust the commitment, reconfirm from the caller or escalate to human come naturally but hard to define a deterministic paralinguistic to build accountability, which is probably very wide.

How are you modeling tone in our voice-agents? I see recent TTS models which accept meaningful tags producing great sounding speech, how do we control it ? Does it account for input user's tone.

How does your ASR model captures the tone or there are some good services / models / tools to capture tone. and how do you use it downstream ?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency. Is there a good duplex model which provides transparency in underlying signals beyond just text.

The moment you transcribe to text, you lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone. Those are the signals that tell you whether to trust the commitment, escalate, or verify identity. Is anyone keeping the paralinguistic layer (hesitation, emotion, speaker identity) as structured data instead of dropping it at the mic, and what do you do with it downstream?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency.

The moment you transcribe to text, you lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone.

Those are the signals that tell you whether to trust the commitment, escalate, or verify identity.

Is anyone keeping the paralinguistic layer (hesitation, emotion, speaker identity) as structured data instead of dropping it at the mic, and what do you do with it downstream?

Identical tool-using agent, same tasks, once reading text and once over a real speech pipeline. Text passed ~60%, voice ~20%.

The reasoning was fine, it made the right tool calls; it lost because recognition botched spelled-out identifiers: order IDs, postcodes, the customer's own name. Cheap fixes that beat a bigger model: retry the identifier once, read it back digit by digit, and prefer IDs you can validate against a known set.

Anyone else measuring service providers for voice and text separately? I feel, the gap is bigger than people admit.

The moment you transcribe to text, you lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone. Those are the signals that tell you whether to trust the commitment, escalate, or verify identity. Is anyone keeping the paralinguistic layer (hesitation, emotion, speaker identity) as structured data instead of dropping it at the mic, and what do you do with it downstream?

Moreover end-2-end Duplex models limits it to trained data scenarios without no transparency.