unsloth-orpo
Unsloth-orpo facilitates one-step preference alignment using Odds Ratio Preference Optimization (ORPO). Unlike DPO, which requires a separate reference model, ORPO incorporates a penalty for disfavored generations directly into the training process, making it more efficient and faster.
npx skills add majiayu000/claude-skill-registry --skill unsloth-orpo-cuba6112-skillfactory-050d4629 --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
--- name: unsloth-orpo description: One-step preference alignment using Odds Ratio Preference Optimization (ORPO) (triggers: ORPO, preference optimization, alignment, ORPOTrainer, log_odds_ratio, binary preference). --- ## Overview Unsloth-orpo facilitates one-step preference alignment using Odds Ratio Preference Optimization (ORPO). Unlike DPO, which requires a separate reference model, ORPO incorporates a penalty for disfavored generations directly into the training process, making it more efficient and faster. ## When to Use - When aligning models with human preferences (Chosen vs. Rejected) in a single step. - When seeking an efficient alternative to DPO or RLHF. - When training on preference datasets like `ultrafeedback_binarized`. ## Decision Tree 1. Do you have a reference model? - No: Use ORPO for one-step alignment. 2. Is the model diverging from language patterns? - Yes: Monitor `nll_loss` (the SFT component) and adjust learning rate. 3. Is the preference gap widening? - Monitor `rewards/margins` and `log_odds_ratio` in logs. ## Workflows 1. **One-Step Preference Alignment**: Load a model in 4-bit, apply LoRA, and initialize `ORPOTrainer` with a low learning rate (approx.
- Overview
- When to Use
- Decision Tree
- Workflows
- Non-Obvious Insights
- Evidence
- Scripts
- Dependencies
- References
What does the unsloth-orpo skill do?
Unsloth-orpo facilitates one-step preference alignment using Odds Ratio Preference Optimization (ORPO). Unlike DPO, which requires a separate reference model, ORPO incorporates a penalty for disfavored generations directly into the training process, making it more efficient and faster.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill unsloth-orpo-cuba6112-skillfactory-050d4629 --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.