grpo
Group Relative Policy Optimization for reinforcement learning from human feedback. Covers GRPOTrainer, reward function design, policy optimization, and KL divergence constraints for stable RLHF training. Includes thinking-aware reward patterns.
npx skills add majiayu000/claude-skill-registry --skill grpo --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Group Relative Policy Optimization (GRPO) ## Overview GRPO is a reinforcement learning method for LLM alignment. It generates multiple completions per prompt, scores them with a reward function, and optimizes the policy to favor higher-reward responses using relative policy gradients. This skill includes patterns for training thinking/reasoning models. ## Quick Reference | Component | Purpose | |-----------|---------| | `GRPOTrainer` | RL trainer for policy optimization | | `GRPOConfig` | Training hyperparameters | | `reward_funcs` | Reward function(s) for scoring | | `completion_ids` | Token IDs passed to reward functions (no re-tokenization) | | `beta` | KL penalty coefficient (0.1 typical) | | `num_generations` | Completions per prompt (2-4) | | `learning_rate` | 1e-5 (10x lower than SFT) | | Token ID 151668 | `</think>` boundary for Qwen3-Thinking models | ## Critical Environment Setup ```python import os from dotenv import load_dotenv load_dotenv() # Force text-based progress in Jupyter os.environ["TQDM_NOTEBOOK"] = "false" # CRITICAL: Set BEFORE importing unsloth/TRL os.environ['ACCELERATE_MIXED_PRECISION'] = 'bf16' ``` ## Critical Import Order ```python # CRITICAL: Import
- Overview
- Quick Reference
- Critical Environment Setup
- Critical Import Order
- GRPO Concepts
- How GRPO Works
- Key Differences from PPO
- Setup
- Load Model
- Apply LoRA
- Dataset Format
- Reward Functions
- Simple Reward Function
- LLM-as-Judge Reward
What does the grpo skill do?
Group Relative Policy Optimization for reinforcement learning from human feedback. Covers GRPOTrainer, reward function design, policy optimization, and KL divergence constraints for stable RLHF training. Includes thinking-aware reward patterns.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill grpo --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.