Meet Kavi — a voice agent that connects your apps and remembers faces (built on AIUI)
Hi everyone 👋 I've been building Kavi, an AIUI .aix agent for Rokid Glasses, and wanted to share it with the community.
📦 Code & roadmap: https://github.com/zrg-team/rokid-personal-assistant
🟢 WHAT IT IS
Say "Kavi" and ask in plain language - Kavi plans a tool call, shows a heads-up card, and speaks a short answer. Two ideas drive it:
🔌 Connect your apps.
Every service is a "connection" brokered by Composio, authorized once on your phone and never stored on the glasses. Google Calendar is wired end to end ("what's on today?", "am I free tomorrow?", "when does my flight start?"); Gmail and Slack ride the same rails. Say "Kavi [app] [action]", or set your own shortcut (say "mail" instead of "gmail").
🧠 Remember people by face.
Glance at someone and say "Kavi halo" to remember them; later ask "who is this?" and Kavi answers with their name, your notes, and any meeting you share that day.
🔒 HOW IT WORKS (AND PRIVACY)
The glasses stay a thin client - capture, consent, render, speak. The heavy lifting (face recognition, sign-in, the connections proxy) runs in cloud functions, so no model or secret ever ships to the device. Sign-in is passwordless: a short code on the HUD, opened on your phone, and the glasses finish on their own - no app to install, no password on the glasses. Faces are kept as vectors (never raw photos), scoped to you; a stranger's face is never saved.
There's also a phone console to manage your connections, the people Kavi remembers, and your voice shortcuts from one place - and stay signed in with a passcode.
🔑 SIGNING IN - AND AN HONEST SECURITY NOTE
Sign-in is passwordless and device-first. Your phone does a normal Google sign-in (that Google account is your identity), you type the short code shown on the glasses, and a per-device token is minted. No password is ever entered on the glasses.
SIGN-IN / PAIR FLOW (device-authorization, RFC 8628 style)
I want to be upfront: this is pragmatic, not bank-grade. Honestly -
- The short pairing code is a small, human-sized secret. A confirm word shown on BOTH screens blocks phishing, and per-IP rate limits bound guessing - but it is not a cryptographic handshake.
- The app key baked into the glasses app is NOT a secret. It ships inside the .aix and the public console JavaScript, so treat it as a build tag / noise filter / rotation switch, not authentication.
- The public "anon" key can do nothing on its own - every table sits behind row-level security, and only the server-side cloud functions hold the service-role key.
- Requests are not individually device-signed yet. That is the next hardening step (per-device signed requests via WebAuthn / crypto.subtle), alongside passkey / Face-ID unlock for the console.
So your app grants live in Composio server-side, faces are vectors scoped to your account, and nothing secret sits on the glasses - but the transport-level auth is deliberately simple today, and hardening it is squarely on the roadmap. If you have done AIUI device auth well, I would love pointers.
🗺️ THE PLAN
- More connections (Notion, Tasks, Contacts, ...) - each is a small config add, not a new auth flow
- Passkey / Face-ID unlock for the console
- Richer people memory - linking contacts to faces
Full roadmap and source: https://github.com/zrg-team/rokid-personal-assistant
⚠️ STATUS - IN REVIEW, NOT YET APPROVED
Kavi is currently submitted for review and has NOT yet been approved by Rokid, so it isn't in the store yet. I'm sharing it here while it's in review to get early feedback:
- Which connections would you want first?
- Any AIUI / Ink gotchas I should watch out for?
Happy to answer anything about the build - the AIUI Ink pages, the device sign-in flow, or the face pipeline. Thanks for reading! 🙏
