fsdp

PyTorch FSDP — sharding strategies, mixed precision, activation checkpointing, auto_wrap_policy, FSDP2 migration, async checkpointing (DCP), torch.compile integration, and HuggingFace integration. Use when training models too large for a single GPU with FSDP. NOT for DeepSpeed (see deepspeed).