The choice between FSDP and DeepSpeed used to be obvious — pick DeepSpeed if you needed ZeRO-3 efficiency, pick FSDP if you wanted to stay within PyTorch. It is no longer obvious. Both frameworks have converged on similar capabilities, and the differences that remain are subtle and workload-dependent.
A side-by-side benchmark on a 30B model, across three clusters and four hardware generations.