unsloth-long-context
Unsloth enables training on extreme context lengths (up to 89K+ on a single 80GB GPU) by utilizing manually derived Triton kernels for RoPE and attention. It optimizes memory usage by a further 30% compared to Flash Attention 2, allowing for 4x longer context windows.
npx skills add majiayu000/claude-skill-registry --skill unsloth-long-context-cuba6112-skillfactory-c58e9994 --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
--- name: unsloth-long-context description: Training models on extended context lengths using optimized RoPE scaling and memory-efficient attention kernels. Triggers: long context, max_seq_length, rope scaling, large context window, flex attention. --- ## Overview Unsloth enables training on extreme context lengths (up to 89K+ on a single 80GB GPU) by utilizing manually derived Triton kernels for RoPE and attention. It optimizes memory usage by a further 30% compared to Flash Attention 2, allowing for 4x longer context windows. ## When to Use - When training on long documents, codebases, or books. - When building models that require large retrieval windows or multi-document reasoning. - When standard Flash Attention 2 results in OOM errors on long sequences. ## Decision Tree 1. Is context > 32K? - Yes: Set `use_gradient_checkpointing = 'unsloth'` (mandatory for stability). 2. Are you seeing quality degradation on long context? - Yes: Ensure your dataset includes samples with long-range dependencies and adjust RoPE base frequency. 3. Using A100/H100 80GB? - Yes: You can push context lengths toward 89K tiers. ## Workflows ### Setting Up Extreme Context Training 1. Load model with hig
- Overview
- When to Use
- Decision Tree
- Workflows
- Setting Up Extreme Context Training
- RoPE Scaling Configuration
- Non-Obvious Insights
- Evidence
- Scripts
- Dependencies
- References
What does the unsloth-long-context skill do?
Unsloth enables training on extreme context lengths (up to 89K+ on a single 80GB GPU) by utilizing manually derived Triton kernels for RoPE and attention. It optimizes memory usage by a further 30% compared to Flash Attention 2, allowing for 4x longer context windows.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill unsloth-long-context-cuba6112-skillfactory-c58e9994 --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.