I built a children's storytelling app with real-time audio/text sync in Flutter, here's what I learned
Long time lurker, first post. Been working on Stela for the past few weeks and figured this community would appreciate the technical side of it more than anyone.
Stela is a children's storytelling app where audio and text stay in sync as you read think YouTube Music lyrics view but for story narration.
The part that actually hurt: word-boundary TTS sync
Getting the text to highlight exactly as each word is spoken (not a sentence ahead, not delayed) was the most frustrating part of this whole build. I used flutter_tts for word-boundary callbacks and custom shader masks to handle the highlight effect. The offset tracking through the full story text had to account for how I split the text into lines, sentence-ending punctuation with regex, otherwise the char positions would drift.
Architecture
Clean Architecture with Flutter + Provider. Each layer is independent:
- Data: Firebase + TTS service (swapped between flutter_tts and ElevenLabs during dev)
- Domain: use cases, repository contracts, entities
- Presentation: providers coordinate audio state, playback position, and UI simultaneously
The hardest part wasn't any one feature, it was keeping the layers clean as the feature set grew. Audio state bleeds into everything if you're not disciplined about it.
Other stuff I built:
- Streak tracking + badge engine (data-driven badge rules so adding a new badge is one line)
- Reading stats (read time, genres, stories completed)
- Floating pill nav bar
- Story detail screen with scroll-to-active-line behaviour
What I'd do differently:
I went back and forth on TTS providers mid-build which cost me time. ElevenLabs has a free web interface but library voices are paywalled on the API, I learned that the hard way. Google Cloud TTS Neural2 voices are the move if you want quality without paying immediately.
Happy to go deep on any of the implementation details. The audio sync and the clean arch setup for Firebase specifically are things I spent a lot of time on.
What are you all using for TTS in Flutter projects?