huggingface-accelerate
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
npx skills add OpenRaiser/NanoResearch --skill accelerate --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# HuggingFace Accelerate - Unified Distributed Training ## Quick start Accelerate simplifies distributed training to 4 lines of code. **Installation**: ```bash pip install accelerate ``` **Convert PyTorch script** (4 lines): ```python import torch + from accelerate import Accelerator + accelerator = Accelerator() model = torch.nn.Transformer() optimizer = torch.optim.Adam(model.parameters()) dataloader = torch.utils.data.DataLoader(dataset) + model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader) for batch in dataloader: optimizer.zero_grad() loss = model(batch) - loss.backward() + accelerator.backward(loss) optimizer.step() ``` **Run** (single command): ```bash accelerate launch train.py ``` ## Common workflows ### Workflow 1: From single GPU to multi-GPU **Original script**: ```python # train.py import torch model = torch.nn.Linear(10, 2).to('cuda') optimizer = torch.optim.Adam(model.parameters()) dataloader = torch.utils.data.DataLoader(dataset, batch_size=32) for epoch in range(10): for batch in dataloader: batch = batch.to('cuda') optimizer.zero_grad() loss = model(batch).mean() loss.backward() optimizer.step() ``` **With Accelerate** (4 lines added):
- Quick start
- Common workflows
- Workflow 1: From single GPU to multi-GPU
- Workflow 2: Mixed precision training
- Workflow 3: DeepSpeed ZeRO integration
- Workflow 4: FSDP (Fully Sharded Data Parallel)
- Workflow 5: Gradient accumulation
- When to use vs alternatives
- Common issues
- Advanced topics
- Hardware requirements
- Resources
pip install accelerate accelerate launch train.py accelerate config Single GPU Multi-GPU (8 GPUs) accelerate launch --multi_gpu --num_processes 8 train.py Multi-node accelerate launch --multi_gpu --num_processes 16 \ train.py accelerate launch --config_file deepspeed_config.json train.py
What does the huggingface-accelerate skill do?
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
How do I install it?
Run `npx skills add OpenRaiser/NanoResearch --skill accelerate --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From OpenRaiser/NanoResearch, a repository with 1,480 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.