u/ivan_digital

I maintain the Soniqo speech stack, and I built this prototype to test whether a useful mobile voice agent really needs a large general-purpose model.

The complete voice-to-action pipeline runs on a Galaxy S23 Ultra:

Silero VAD → Parakeet-EOU 120M STT → FunctionGemma 270M → Android action → Pocket TTS

After the initial model download, speech recognition, action selection and speech synthesis all run locally. No recorded audio or transcript is sent to a server. The complete pipeline uses approximately 1.2 GB of memory on my S23 Ultra.

It currently supports:

• Looking up a contact or phone number
• Opening the Android dialer for a contact
• Dialling a number spoken by the user
• Listing or searching music stored on the device
• Playing and stopping local music
• Setting the media volume
• Explaining which actions are currently available

Calls are not placed silently—the agent opens Android’s dialer so the user remains in control.

FunctionGemma is paired with a 9.5 MB adapter trained on a compact command-to-action format. It produces one structured tool call rather than an open-ended conversational response.

One useful design choice was filtering the available tools according to the current device state. For example, stop_music is only presented to the model while music is playing. This prevents some invalid actions before generation instead of asking the model to reason around them.

The model is also not trusted to invent outcomes. Contact and music searches are checked against real device data. The spoken confirmation comes from the result returned by Android rather than what the model predicted would happen.

This is still a deliberately narrow prototype, not a replacement for Google Assistant. The experiment is about whether small local models can provide fast, inspectable device control without routing private speech through a cloud service.

Demo: https://www.youtube.com/watch?v=7L7_Uvvxtv0 Source and signed APK: https://github.com/soniqo/speech-android

I’m curious where others would draw the boundary: keep extending a constrained action router, or introduce a larger planner once commands require multiple steps?

I maintain speech-android (Apache-2.0). We published a demo voice assistant that runs the whole loop on the device: local VAD, local streaming speech recognition, a 270M function-calling model that picks the action, and local TTS for the answer. After the one-time model download (~560 MB) it works in airplane mode — there is no server side, no Google services, and no trackers.

Runs on arm64 phones, Android 8+, using about 1.2 GB of RAM while active (measured on a Galaxy S23).

90-second demo: https://youtu.be/7L7_Uvvxtv0 APK (GitHub releases) + source: https://github.com/soniqo/speech-android

I built Speech Studio, an Apache 2.0 desktop app for local voice cloning and multi-speaker script rendering.

Source: https://github.com/soniqo/speech-studio

It is built on the speech-swift library/CLI: https://github.com/soniqo/speech-swift

I also published a benchmark for several local voice-cloning models across English, German, Arabic, Spanish, and Chinese:

https://www.soniqo.audio/blog/voice-cloning-benchmarks

The benchmark includes reference audio, generated audio, speaker similarity, WER/CER, generated audio length, and RTF.

I put together a dataset-backed benchmark for local voice cloning models:


- OmniVoice int8
- Chatterbox Multilingual fp16
- VoxCPM2 bf16
- Fish Audio S2 Pro fp16


It uses Google FLEURS test clips as references, then reports speaker similarity, WER/CER, generated audio length, and RTF. I also included the reference audio and generated clips for every row, so the table is not just numbers.


Post:
https://www.soniqo.audio/blog/voice-cloning-benchmarks


The result that surprised me most: OmniVoice was the best all-around row set in this run, but the language-by-language behavior is more interesting than the aggregate. VoxCPM2 was strong on Arabic speaker match; Fish Audio had strong German/Arabic similarity but slower RTF; Chatterbox looked good on Arabic/Spanish but I am not benchmarking Chinese until the Swift tokenizer path is ready.


I maintain the Soniqo speech stack, so this is self-promo, but the benchmark is meant to be useful/reproducible rather than a launch post.


Speech Studio is the open-source desktop app built on the same stack:


https://www.soniqo.audio/speech-studio
https://github.com/soniqo/speech-studio


What model/language should I add next?

NVIDIA released Nemotron-3.5-ASR-Streaming-0.6B last month. Same FastConformer + RNN-T family as their earlier English-only streaming model, but trained on 40 language-locales and conditioned by a prompt kernel that injects a one-hot language slot into every encoded frame — so the same 600M weights serve every language. Ported it to Apple Silicon end-to-end:

• CoreML INT8 bundle that routes through the Apple Neural Engine (encoder + RNN-T decoder + joint network)
• MLX bf16 / 8-bit / 4-bit bundles for GPU-resident inference

WER (FLEURS test, 50 samples/language, M5 Pro):

*char-level scoring, matching NVIDIA's CJK methodology.

INT8 palletization, MLX bf16, and MLX 8-bit all stay within ±0.3 pp WER of the fp32 PyTorch reference. MLX 4-bit costs ~6 pp on average in exchange for the smallest disk (473 MB) and streaming RSS (747 MB) — useful when disk or RAM is the binding constraint.

Architecture preserved — exported through EncDecRNNTBPEModelWithPrompt (restored from EncDecHybridRNNTCTCBPEModelWithPrompt since the prompt-RNNT target class hasn't been released yet):

audio → mel (NeMo FilterbankFeatures-equivalent) → 24-layer cache-aware FastConformer encoder (1024 hidden) → prompt kernel: Linear(1152→2048) → ReLU → Linear(2048→1024) (folds one-hot language_mask into every encoded frame) → RNN-T: 2-layer LSTM predictor (640 hidden) + joint over 13 087 BPE

Streaming caches — attention KV [24, 1, 56, 1024], depthwise conv [24, 1, 1024, 8], mel pre_cache — flow chunk-to-chunk so context survives the 320 ms boundaries. Streaming RTF on M5 Pro is 0.068 with the CoreML INT8 bundle (p50 chunk latency 18.6 ms, p99 23.4 ms).

Bit-identical Swift↔Python WER on 5 of 6 languages — to validate the Apple-side numbers I ported Whisper's BasicTextNormalizer + EnglishTextNormalizer + the English number-words state machine to Swift. The lone 0.04 pp Hindi delta traces to a single ANE non-determinism sample, not a port bug.

Bundles (Apache 2.0 SDK; bundles carry NVIDIA's eval license, linked on each model card):

• https://huggingface.co/aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-CoreML-INT8
• https://huggingface.co/aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-MLX-bf16
• https://huggingface.co/aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-MLX-8bit
• https://huggingface.co/aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-MLX-4bit

Repo: https://github.com/soniqo/speech-swift Guide: https://soniqo.audio/guides/nemotron

C++17 runtime for real-time voice agents: VAD-driven turn detection, interruption handling, speech queue with cancel/resume, plus reference model wrappers behind abstract STT / TTS / VAD / LLM interfaces (bring your own backend if you prefer).

Models wired up, all on-device CPU:

- VAD: Silero v5

- STT: Parakeet TDT v3 (batch) · Nemotron Speech Streaming 0.6B (true streaming RNN-T, ~80 ms partials) · Omnilingual ASR CTC-300M (multilingual)

- Diarization: Pyannote Segmentation 3.0 + WeSpeaker ResNet34-LM, composed in pure C++

- TTS: VoxCPM2 (2B, 48 kHz, zero-shot voice cloning) · Kokoro 82M

- Enhancement: DeepFilterNet3

Two interchangeable backends: ONNX Runtime and LiteRT (Google's ai-edge-litert). Both CPU today; CUDA / TensorRT EP just landed on the ONNX path (gated, default off). Runs on Linux x86_64 + aarch64, Windows x86_64, Android. Stable C ABI for FFI (Swift, Kotlin, Python, …). The orchestration core has zero ML dependencies.

https://github.com/soniqo/speech-core

C++17 runtime that composes several open speech models behind a small interface layer:

• Silero VAD → StreamingVAD (4-state hysteresis: silence / pendingSpeech / speech / pendingSilence)
• Parakeet TDT v3 (FastConformer encoder INT8 + decoder-joint FP32 RNN-T state; CTC fallback)
• Nemotron Speech Streaming 0.6B (cache-aware FastConformer + RNN-T, true streaming)
• Omnilingual ASR CTC-300M (Wav2Vec2 + CTC, SentencePiece decode)
• Pyannote Segmentation 3.0 + WeSpeaker ResNet34-LM → constrained agglomerative clustering in pure C++ (no ML-runtime dep)
• VoxCPM2 (2B AR LM + AudioVAE, 48 kHz, zero-shot voice cloning, 4-graph pipeline: text_prefill → token_step ×N → audio_decoder)
• Kokoro 82M, DeepFilterNet3

Two interchangeable backends — ONNX Runtime and LiteRT (libLiteRt from Google's ai-edge-litert wheel) — both CPU today; CUDA / TensorRT EP just landed on the ONNX path (build-flag gated, env-resolved, runtime-probed, CPU fallback). Build the orchestration core alone (zero ML deps) or with either / both backends.

C++17, Apache 2.0, Linux + Windows + Android, stable C ABI for FFI.

https://github.com/soniqo/speech-core

Just open-sourced Speech Studio — a Mac app for voice cloning and multi-speaker dialog generation, runs entirely on-device on Apple Silicon. No API keys, no cloud, no per-character billing.

Drop a reference clip → cloned voice. Drop another → second voice. Write a multi-speaker scene with inline emotion markers like (whispering) or (intense). Hit Synthesize.

30-sec A/B/C blind test vs ElevenLabs: https://youtu.be/EuIU8tOWyzg

(Same voice, three takes — real, Speech Studio local, ElevenLabs cloud.)

• Repo + DMG (Apache 2.0): https://github.com/soniqo/speech-studio
• Docs: https://soniqo.audio/speech-studio
• macOS 15+ on M1/M2/M3/M4

In the comments — feedback welcome, especially on the cloning quality.

Hey r/swift,

I made a short demo video of speech-swift, an open-source Swift library for local speech AI on Apple Silicon.

It shows realtime transcription, speech-to-speech, voice cloning, and local TTS running without a cloud speech API.

The voiceover in the video was generated with the library’s voice cloning / TTS capabilities, using my own voice reference.

Video: https://www.youtube.com/watch?v=x9zgcaW0gUk
Repo: https://github.com/soniqo/speech-swift

Would love feedback from Swift developers.

Most voice input on Android - SpeechRecognizer.createSpeechRecognizer(context) calls — gets routed to Google's network-backed recognizer. I wanted that path to run locally, so I wrote one.

The service hooks the framework's SpeechRecognizer API. Once it's set as the default, any app calling createSpeechRecognizer(context) (no ComponentName) ends up in our pipeline and gets back transcription that never left the device. Pipeline is Silero VAD + Parakeet TDT v3 (114 languages, ~890 MB INT8) on ONNX Runtime with NNAPI.

Honest caveat: Gboard, Samsung Keyboard, and Google Assistant ship their own recognizers and skip the system default. So the default-IME voice button on most phones won't go through this. What does: accessibility tools, custom dictation UIs, and anything calling the framework API directly.

Models download on first use (~1.2 GB) via a foreground WorkManager job so it survives backgrounding. After that, fully offline.

Setup + demo APK: github.com/soniqo/speech-android

audio.soniqo:speech:0.0.9 on Maven Central

Library:

Happy to answer questions about the binder lifecycle, the foreground worker setup, or why SpeechRecognizer is such a tarpit of edge cases.