u/Radianis

Has anyone quantified the actual compute waste from training divergence at scale? Trying to understand how common rollback and restart really is in practice.