Tag
#FSDP
2 articles tagged FSDP.
A research-grade trainer in 400 lines
Training SystemsMost training frameworks are 50,000 lines of code in a trench coat. This is what falls out when you start from FSDP and a will to delete.
FSDP vs DeepSpeed, 2026 edition
Distributed TrainingThe choice used to be obvious. It isn't anymore. A side-by-side on training a 30B model across three clusters and four hardware generations.