rloo
Reinforcement Learning with Leave-One-Out estimation for policy optimization. Covers RLOOTrainer, reward function integration, baseline estimation, and variance reduction techniques for stable RL training. Includes thinking-aware patterns.
Profile →npx skills add majiayu000/claude-skill-registry --skill rloo --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Reinforcement Learning with Leave-One-Out (RLOO) ## Overview RLOO is a reinforcement learning method that uses leave-one-out baseline estimation for variance reduction. Like GRPO, it generates multiple completions per prompt but uses a different baseline computation that can provide more stable gradients. This skill includes patterns for training thinking/reasoning models. ## Quick Reference | Component | Purpose | |-----------|---------| | `RLOOTrainer` | RL trainer with RLOO baseline | | `RLOOConfig` | Training hyperparameters | | `reward_funcs` | Reward function(s) for scoring | | `completion_ids` | Token IDs passed to reward functions (no re-tokenization) | | `num_generations` | Completions per prompt (4 typical) | | `kl_coef` | KL penalty coefficient (0.05, lower than GRPO) | | `learning_rate` | 1e-5 (same as GRPO) | | Token ID 151668 | `</think>` boundary for Qwen3-Thinking models | ## Critical Environment Setup ```python import os from dotenv import load_dotenv load_dotenv() # Force text-based progress in Jupyter os.environ["TQDM_NOTEBOOK"] = "false" # CRITICAL: Set BEFORE importing unsloth/TRL os.environ['ACCELERATE_MIXED_PRECISION'] = 'bf16' ``` ## Critical Import Order
- Overview
- Quick Reference
- Critical Environment Setup
- Critical Import Order
- RLOO Concepts
- How RLOO Works
- Leave-One-Out Baseline
- Comparison with GRPO
- Dataset Format
- Setup
- Load Model
- Apply LoRA
- RLOOTrainer Configuration
- Basic Configuration
What does the rloo skill do?
Reinforcement Learning with Leave-One-Out estimation for policy optimization. Covers RLOOTrainer, reward function integration, baseline estimation, and variance reduction techniques for stable RL training. Includes thinking-aware patterns.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill rloo --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.