Fine-tuned Qwen3-ASR-0.6B on 1,000 hours of Hindi/Hinglish call audio: beats Whisper large-v3, Azure and Google on accented Hindi at half the size (Apache-2.0)
Weights: https://huggingface.co/tryorato
We build voice agents for Indian and Gulf businesses, and every off-the-shelf ASR fell apart on real calls: accented Hindi, constant Hindi-English code-switching, noisy lines. So we fine-tuned Qwen3-ASR-0.6B on roughly 1,000 hours of Hindi, English and Hinglish calling audio.
Setup
- Base: Qwen/Qwen3-ASR-0.6B
- ~0.8B total params, ~0.6B trainable
- Full SFT on decoder and projector, audio tower frozen
- Corpus: Rasa Hindi, Gram Vaani, MUCS, plus proprietary enterprise call data
- LR 1e-5, cosine, warmup 0.03, bf16, max grad norm 1.0, 1 epoch
Gains over base Qwen3-ASR
| Benchmark | Base | Ours | Rel. reduction |
|---|---|---|---|
| Kathbath (read/clean) | 15.24 | 11.49 | 24.6% |
| Gramvaani (rural/noisy telephony) | 39.07 | 37.66 | 3.6% |
| Lahaja (dialects/accents) | 25.09 | 18.68 | 25.5% |
| FLEURS (multilingual) | 19.12 | 16.98 | 11.2% |
Lahaja WER, accents and dialects, where we do best
| System | WER |
|---|---|
| Ours (0.8B) | 18.68 |
| IndicASR M1 (Conformer-L) | 19.40 |
| Google Chirp | 22.30 |
| Azure STT | 28.60 |
| Whisper large-v3 (1.55B) | 32.40 |
| MMS (300M) | 34.40 |
Where we lose, stated up front: IndicWhisper is still ahead of us on Kathbath (10.30 vs 11.49), Kathbath-Hard (12.00 vs 13.21), FLEURS (11.40 vs 16.98) and notably Gramvaani (26.80 vs 37.66). Gramvaani is rural noisy telephony and it's our weakest result; it's the target for v2. ElevenLabs Scribe and Azure also beat us on CommonVoice. We are not claiming SOTA Hindi ASR. We're claiming a small, permissively licensed, self-hostable model that holds up on accented conversational speech against models two to three times its size and against paid APIs.
Methodology caveat: our numbers are self-run; competitor numbers are published results from the AI4Bharat Vistaar and Lahaja suites. Not a perfectly controlled comparison, and I'd rather say that than have someone find it. Happy to share our eval config if anyone wants to reproduce.
Gotcha: load via qwen_asr.Qwen3ASRModel.from_pretrained, not transformers.AutoModel. AutoModel skips the custom decoding layers and throws at runtime.
python
import qwen_asr, torch
wrapper = qwen_asr.Qwen3ASRModel.from_pretrained(
"tryorato/orato-asr-hindi-v1",
dtype=torch.bfloat16,
device_map=None,
attn_implementation="sdpa",
)
wrapper.model = wrapper.model.to("cuda")
result = wrapper.transcribe(audio=(wav, 16000), language="Hindi")
Apache-2.0, use it for whatever. Hindi TTS is next. Questions welcome.