[ESP32][AWS IoT][OpenSource] A robotic hand that copies my gestures over the internet, so technically I can flip someone off from another continent lol :p
I built a 5-finger bionic hand that mirrors my real hand in (almost) real time. The catch: my hand and the robot hand never actually talk to each other. Everything goes through the cloud.
The pipeline is :
- Webcam + MediaPipe tracks 21 landmarks on my hand at 30+ FPS and figures out how bent each finger is.
- That gets turned into a finger-by-finger "power %" and published to AWS IoT Core over MQTT.
- An ESP32 on the other end is subscribed to the same topic, parses the JSON, and drives 5 servos through a PCA9685 PWM controller over I2C.
- Firmware is ESP-IDF + FreeRTOS (Dual-core Operation). One core handles WiFi/MQTT, the other handles the servos, and they talk to each other over a queue.
- End-to-end latency is ~150ms (AWS US East <--> Michigan). No port forwarding, no VPN, the ESP32 dials out to AWS and waits.
Things I learned the fun way:
- Servos draw way more current than an ESP32 can drive. Power them separately or there will be random reboots.
- The thumb needed inverted control logic because thumbs are, anatomically, little contrarians.
It's all open source, hardware, firmware, the Python vision client, wiring diagrams, full setup guide.
Genuinely usable as a capstone project or a base for prosthetics/telepresence research, but mostly I just like playing Stone-Paper-Scissors with it now.
Git Repo: https://github.com/itsashishupadhyay/VisionControlledBionicHand
Bonus: if you've wanted,a working reference for the AWS IoT Device SDK for Embedded C or a hand-rolled PCA9685 driver library, this repo doubles as a decent starting point for both.