https://github.com/alex000kim/nanoRL
nanoRL is the smallest async RL trainer I could write:
the same disaggregated trainer/worker setup that runs on 16 GPUs runs as two pods on your
laptop.
~1,800 lines across 7 files, no Ray, TRL or DeepSpeed.
A lot of ML teams end up with a mix:
some Slurm clusters from the HPC side, a K8s cluster or two, maybe cloud GPUs for overflow. We wrote up how SkyPilot (open source) sits in front of all of them so a job is scheduled wherever there’s free capacity, using the same YAML regardless of backend. This post focuses on the multi-Slurm case but the same setup covers K8s.
https://skypilot.ai/blog/multi-slurm
Disclosure:
I am the author. Happy to answer questions about how the scheduling and failover work.