unsloth-lora
Configuring and optimizing 16-bit Low-Rank Adaptation (LoRA) and Rank-Stabilized LoRA (rsLoRA) for efficient LLM fine-tuning using triggers like lora, qlora, rslora, rank selection, lora_alpha, lora_dropout, and target_modules.
npx skills add majiayu000/claude-skill-registry --skill unsloth-lora --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
## Overview Unsloth optimizes Low-Rank Adaptation (LoRA) by providing 16-bit trainable matrices that allow for efficient fine-tuning without updating all model weights. It supports standard LoRA and Rank-Stabilized LoRA (rsLoRA), utilizing specialized kernels to accelerate training and reduce memory overhead. ## When to Use - When fine-tuning large language models on consumer-grade or limited GPU hardware. - When aiming to match full fine-tuning performance with significantly lower VRAM usage. - When specialized scaling (rsLoRA) is required for higher rank stability. ## Decision Tree 1. Need to update all weights? - Yes: Use [[unsloth-fft]]. - No: Proceed to LoRA. 2. Using high rank (r > 64)? - Yes: Enable `use_rslora = True` for sqrt(r) scaling. - No: Use standard LoRA. 3. Maximizing speed? - Yes: Set `lora_dropout = 0` to enable internal kernel optimizations. ## Workflows ### Optimizing LoRA Architecture 1. Target all 7 major linear layers (q, k, v, o, gate, up, down) to match full fine-tuning performance. 2. Initialize rank (r) between 16 and 32 for general tasks, or up to 128 for complex domain adaptation. 3. Set lora_alpha equal to r or 2*r to maintain aggressive learning whil
- Overview
- When to Use
- Decision Tree
- Workflows
- Optimizing LoRA Architecture
- Configuring Rank-Stabilized LoRA (rsLoRA)
- Non-Obvious Insights
- Evidence
- Scripts
- Dependencies
- References
What does the unsloth-lora skill do?
Configuring and optimizing 16-bit Low-Rank Adaptation (LoRA) and Rank-Stabilized LoRA (rsLoRA) for efficient LLM fine-tuning using triggers like lora, qlora, rslora, rank selection, lora_alpha, lora_dropout, and target_modules.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill unsloth-lora --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.