unsloth-cpt
Unsloth-cpt provides specific optimizations for Continued Pretraining (CPT) and domain adaptation. It addresses the critical need for training embedding layers and language modeling heads while stabilizing the training process using Rank Stabilized LoRA (rsLoRA) and differentiated learning rates.
Profile →npx skills add majiayu000/claude-skill-registry --skill unsloth-cpt-cuba6112-skillfactory-cf577731 --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
--- name: unsloth-cpt description: Strategies for continued pretraining and domain adaptation in Unsloth (triggers: continued pretraining, CPT, domain adaptation, lm_head, embed_tokens, rsLoRA, embedding_learning_rate). --- ## Overview Unsloth-cpt provides specific optimizations for Continued Pretraining (CPT) and domain adaptation. It addresses the critical need for training embedding layers and language modeling heads while stabilizing the training process using Rank Stabilized LoRA (rsLoRA) and differentiated learning rates. ## When to Use - When teaching a model a new language or highly specialized domain (e.g., legal, medical). - When updating the `embed_tokens` or `lm_head` layers. - When using high LoRA ranks (e.g., r=256) which can become unstable without rsLoRA. ## Decision Tree 1. Are you training on a new domain with unique vocabulary? - Yes: Include `lm_head` and `embed_tokens` in `target_modules`. 2. Are you using a LoRA rank > 64? - Yes: Set `use_rslora = True`. 3. Are you training embeddings? - Yes: Set `embedding_learning_rate` to 1/10th of the standard learning rate. ## Workflows 1. **New Language Adaptation**: Load the base model and configure `get_peft_model` to
- Overview
- When to Use
- Decision Tree
- Workflows
- Non-Obvious Insights
- Evidence
- Scripts
- Dependencies
- References
What does the unsloth-cpt skill do?
Unsloth-cpt provides specific optimizations for Continued Pretraining (CPT) and domain adaptation. It addresses the critical need for training embedding layers and language modeling heads while stabilizing the training process using Rank Stabilized LoRA (rsLoRA) and differentiated learning rates.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill unsloth-cpt-cuba6112-skillfactory-cf577731 --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.