dspy-evaluation-suite
This skill should be used when the user asks to "evaluate a DSPy program", "test my DSPy module", "measure performance", "create evaluation metrics", "use answer_exact_match or SemanticF1", mentions "Evaluate class", "comparing programs", "establishing baselines", or needs to systematically test and measure DSPy program quality with custom or built-in metrics.
npx skills add majiayu000/claude-skill-registry --skill dspy-evaluation-suite --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# DSPy Evaluation Suite ## Goal Systematically evaluate DSPy programs using built-in and custom metrics with parallel execution. ## When to Use - Measuring program performance before/after optimization - Comparing different program variants - Establishing baselines - Validating production readiness ## Related Skills - Use with any optimizer: [dspy-bootstrap-fewshot](../dspy-bootstrap-fewshot/SKILL.md), [dspy-miprov2-optimizer](../dspy-miprov2-optimizer/SKILL.md), [dspy-gepa-reflective](../dspy-gepa-reflective/SKILL.md) - Evaluate RAG pipelines: [dspy-rag-pipeline](../dspy-rag-pipeline/SKILL.md) ## Inputs | Input | Type | Description | |-------|------|-------------| | `program` | `dspy.Module` | Program to evaluate | | `devset` | `list[dspy.Example]` | Evaluation examples | | `metric` | `callable` | Scoring function | | `num_threads` | `int` | Parallel threads | ## Outputs | Output | Type | Description | |--------|------|-------------| | `score` | `float` | Average metric score | | `results` | `list` | Per-example results | ## Workflow ### Phase 1: Setup Evaluator ```python from dspy.evaluate import Evaluate evaluator = Evaluate( devset=devset, metric=my_metric, num_threads=8, displ
- Goal
- When to Use
- Related Skills
- Inputs
- Outputs
- Workflow
- Phase 1: Setup Evaluator
- Phase 2: Run Evaluation
- Built-in Metrics
- answerexactmatch
- SemanticF1
- Custom Metrics
- Basic Metric
- Multi-Factor Metric
What does the dspy-evaluation-suite skill do?
This skill should be used when the user asks to "evaluate a DSPy program", "test my DSPy module", "measure performance", "create evaluation metrics", "use answer_exact_match or SemanticF1", mentions "Evaluate class", "comparing programs", "establishing baselines", or needs to systematically test and measure DSPy program quality with custom or built-in metrics.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill dspy-evaluation-suite --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.