lora-qlora-recipes
Configure LoRA and QLoRA supervised fine-tuning with current best-practice hyperparameters. Use when writing or reviewing a LoRA/QLoRA training configuration, choosing rank/alpha/target modules, or deciding between LoRA, QLoRA, and full fine-tuning.
Profile →npx skills add wshobson/agents --skill lora-qlora-recipes --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# LoRA & QLoRA Recipes This skill assumes the routing decision already happened — `finetuning-method-selection` should have already pointed here because the data shape is demonstrations (SFT), not preference pairs or a verifiable reward signal. What follows is the current best-practice recipe for configuring the adapter itself: which modules to target, how to size rank and alpha, what learning rate to use, and when QLoRA buys real headroom versus when it just adds risk. Dataset preparation and quality checks are a separate concern — see `dataset-curation`. **Input:** a routing decision (SFT via LoRA/ QLoRA) plus a target size class. **Output format:** a validated adapter config — the kwarg values below, not free-form advice — that `llm-finetuning-training-engineer` consumes directly when it generates a runnable script. ## The Reference Recipe The reference recipe is "LoRA Without Regret" (Thinking Machines/Schulman, 2025-09), now the settled convention for LoRA/QLoRA SFT. ### Target Modules Target **all-linear** modules, not just attention: ```python target_modules = [ "q_proj", "k_proj", "v_proj", "o_proj", # attention "gate_proj", "up_proj", "down_proj", # MLP — matters most ] ``
- The Reference Recipe
- Target Modules
- Alpha and Learning Rate
- Rank by Task
- Effective Batch Size
- Unsloth Defaults
- LoRA vs QLoRA vs Full FT
- Failure Modes
- References
python -c "import torch; print(torch.cuda.is_bf16_supported())"
What does the lora-qlora-recipes skill do?
Configure LoRA and QLoRA supervised fine-tuning with current best-practice hyperparameters. Use when writing or reviewing a LoRA/QLoRA training configuration, choosing rank/alpha/target modules, or deciding between LoRA, QLoRA, and full fine-tuning.
How do I install it?
Run `npx skills add wshobson/agents --skill lora-qlora-recipes --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From wshobson/agents, a repository with 38,479 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.