Before swapping speech models, how do you check whether the recogniser even got the last bit of audio?
I'm thinking capture timestamps, endpoint time and the last frame sent to ASR. Then replay the same audio offline.
Is there a simpler way people debug this? Feels like it's easy to blame the wrong layer.
Same sentence, same volume, don't show the provider name. Seems like a start.
But a voice can sound great for one line and get pretty weird halfway through a long answer. Do you test short clips and conversations seperately?
If you've run listening tests, what made the biggest difference to the results?
"Tuesday, sorry Thursday" is not the same as "Tuesday Thursday". And losing "don't" is a pretty big edit.
What do you put in tests for the cleanup step? I'd start with corrections, names and numbers, but there's probably some obvious case I'm missing.
What broke for you that a readability check would've completely missed?
Example: "send 15, sorry 50 units". ASR gets the words right, but the next bit of the system hangs onto 15.
The transcript score looks fine. The order is wrong.
Are people checking the final saved fields seperately, or does your speech eval stop at the transcript? Curious how you count a clarification too, since asking again might be the right answer.