
Custom AI voice assistant made with Home Assistant
I wanted to share a build, because I still can't quite believe how well it works and how easy Home Assistant made it to put together.
I wanted a voice assistant that could not only control my smart home, but also be conversationally intelligent, have its own personality, and actually sound like a real voice.
I assembled a mic and speaker and connected them via an ESP32 with ESPHome to Home Assistant. The mic picks up what I say out loud in the room. Then in Home Assistant's voice assistant tools, I added a custom wake word and connected the system to ChatGPT to handle my commands, comments, or questions. The tools even allow for the style of the response to be customised to give it a specific flair or personality. The response comes back from the AI and is routed via ElevenLabs to convert the text to speech, which then plays out loud through the device speaker.
The assistant is also exposed to all my smart devices, so it can control them when I ask and it's intelligent enough to read between the lines, so I don't need to specifically command every action. I can say "The kitchen is messy" and it knows to send my robot vacuum to the kitchen, rather than me needing to say "Send the robot vacuum to clean the kitchen".
And because I gave it a sarcastic personality, it'll usually have a dig at me first something like "Oh, the kitchen's messy is it? What a shocker."
I also use automations to trigger the voice assistant rather than always needing to activate it with a wake word. This means it can greet me when I come home unprompted, when my door sensor is activated.
The electronics parts are so cheap and easy to put together that I've got a device in every room for complete coverage.
I filmed the whole build if anyone wants to see it come together: https://youtu.be/p02rbeQ2Oe8