AI Voice Chatbot
calling out all the experts regarding AI Voice Chatbot.
Asking for some tips or ideas.
My Architecture - STT (Eleven Labs) -> LLM -> TTS(Eleven Labs)
Issue (according to my managers & CTO)-
- STT should be always accurate.
Our use case is only English.
Our product - Coffee based Voice Assistant. User can ask questions regarding coffee techniques, products etc.
Currently, issues are sometimes text transcribed by STT is not exactly what user said. I have added a parameter which eleven labs supports (Keyterm prompting - it charges 20% premium for keyterms) and in that parameters I have added some coffee products name and chatbot name so that stt model is biased toward those keywords and hence send accurate spelling for those keywords in final transcription.
Apart from this, there is still a case where transcription is not 100% or all of the time accurate.
There are three issues primarily:
Sometimes one or two words dropped from sentence spoken by user.
Spelling issues or words mismatch sometimes.
VAD (currently it is server driven 1.5 sec) but I have read about End of Turn Detection models which we can use for this.
So how to solve this?
I am also open to try different architecture or different STT model - local/cloud anything will do).