pytorch-fsdp

What it does

Provides expert guidance for developing with PyTorch FSDP, covering features like parameter sharding, mixed precision, CPU offloading, and FSDP2, and targets users working with distributed training in PyTorch.

How it works

• Triggered when working with pytorch-fsdp, asking about its features, APIs, solutions, debugging, or best practices.
• Presents patterns and API references for distributed training components (e.g., Join context manager, Joinable, JoinHook) and PyTorch distributed APIs, illustrating how to use distributed primitives, initialization, and context management in the FSDP context.
• Describes example APIs and classes, including examples of usage and initialization steps, to facilitate understanding of distributed execution and synchronization mechanisms in PyTorch.

When to use it

• Use when working with pytorch-fsdp
• Use when asking about pytorch-fsdp features or APIs
• Use when implementing pytorch-fsdp solutions
• Use when debugging pytorch-fsdp code
• Use when learning pytorch-fsdp best practices

What it can touch

• Tools: claude-code
• References to PyTorch distributed APIs such as torch.distributed.init_process_group and related helpers, as well as distributed training concepts like DDP and join context management. (All tool references are as described in the skill’s content.)

Caveats

• The content includes detailed API descriptions and code-like examples that outline behavior of the Join context manager and distributed backends. Users should consult official PyTorch documentation for up-to-date semantics and compatibility notes, especially around initialization and environment configurations.