Functional Odradek from Death Stranding using Pi 5 8GB and AI HAT+2
Here's my latest project, an embodied local AI agent based on the Odradek scanner from the Death Stranding games.
It uses a Pi 5 (8GB), v3 camera module, and AI HAT+2 (Hailo-10H) to control a robotic arm and track objects. Under the hood I used openWakeWord + Whisper to transcribe voice prompts, and then use Qwen3 1.7B Instruct to call a tool that maps the prompts to COCO ids. These get handed off to YOLOv11n to do the object detection.
If you're familiar with the games, the Odradek tracks invisible ghostly figures called BTs so I thought it would be fun to track real life BTs - Bluetooth identifiers. There's a XIAO ESP32-C6 in the head of the tracker that controls the actuator, motor, and lights, but it also scans for BT signals from AI glasses similar to the Nearby Glasses app.
Originally, I wanted to use the BT detection to signal the agent to start looking for people wearing glasses but was disappointed to find the models supported on the Hailo-10H lack the post processing capabilities of similar models available on the older AI HAT+ (Hailo-8). Would love to hear tips for getting this working.