RL-based yaw control for suspended payloads — feedback wanted
I’ve been developing a yaw control system for suspended payloads using reinforcement learning.
Four thrusters are mounted on the suspended platform, and an SAC policy controls them at 3 Hz. Payload yaw is estimated from a camera mounted on the platform.
I started with a fixed 90° target, then extended the controller to variable targets from -90° to +90° in 5° increments. I trained separate policies for three payloads: an H-steel beam, a plate, and a truss.
One interesting result was that larger target angles were not necessarily harder to control. I also found that the learned use of thrust differed considerably between payloads. The H-steel policy tends to use relatively low average thrust with short high-output bursts, while the truss uses higher thrust more continuously.
I also modified the training/reward design to reduce unnecessary thruster output after reaching the target.
The whole system runs at 3 Hz on a Raspberry Pi, using ONNX models for both yaw estimation and control.
The attached video shows the truss payload controlling four target yaw angles (-90°, -45°, +45°, +90°) simultaneously.
Project results and videos:
https://rindajones.github.io/payload-yaw-control/en/
I’m particularly interested in criticism from people working on robotics or control:
- Would you approach this problem with RL at all?
- What disturbances or dynamics would you consider essential for the next step?
- What would you want to see before considering this relevant to a real suspended-load system?
Any criticism is welcome.