metric-design
Use when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining multiple metrics into a composite score, or building an automated evaluation pipeline. Also use when the user mentions grader selection, metric design, judge prompt engineering, rubric design, evaluation pipeline code, or "how to evaluate [X] automatically." Outputs executable OpenJudge pipeline code.
Profile →npx skills add agentscope-ai/OpenJudge --skill 02-metric-design --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Metric Design Select, configure, and combine evaluation graders into a working pipeline. You choose the right tool for each evaluation dimension — from zero-cost code checks to LLM judges — and produce executable `GradingRunner` code that runs on OpenJudge. > **Requires OpenJudge** (`pip install py-openjudge`). This skill is intentionally > SDK-centric — grader selection, `GradingRunner`, and aggregators are OpenJudge APIs. The > design/decision logic still applies if you use another harness; only the code does not. ## When to Activate - User has eval dimensions/principles but doesn't know which grader type to use - User wants to write an LLM-as-judge prompt for a specific failure mode - User needs a composite score combining multiple evaluation dimensions - User wants to auto-generate graders from labeled data instead of writing them manually - User's current evaluation is all LLM-based and too expensive/too slow ## Checklist You MUST create a task for each item and complete them in order: 1. **Select grader types** — per dimension, pick the right grader class 2. **Create custom graders** — write judge prompts (4-component) or function graders 3. **Auto-generate if applicable**
- When to Activate
- Checklist
- Step 1: Select Grader Type Per Dimension
- Grader Selection Cheat Sheet
- Step 2: Create Custom Graders
- LLMGrader: The Four-Component Template
- Complete LLMGrader Code
- FunctionGrader: Deterministic Checks
- Step 3: Auto-Generate Graders (Cold Start)
- Zero-shot: SimpleRubricsGenerator
- Data-driven: IterativeRubricsGenerator
- Step 4: Anti-Pattern Scan
- Step 5: Build Pipeline Code
- Weight Design Principle
What does the metric-design skill do?
Use when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining multiple metrics into a composite score, or building an automated evaluation pipeline. Also use when the user mentions grader selection, metric design, judge prompt engineering, rubric design, evaluation pipeline code, or "how to evaluate [X] automatically." Outputs executable OpenJudge pipeline code.
How do I install it?
Run `npx skills add agentscope-ai/OpenJudge --skill 02-metric-design --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From agentscope-ai/OpenJudge, a repository with 764 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.