eval-harness
Evaluation harness for testing agent and skill quality through structured benchmarks, regression tests, and quality scoring.
npx skills add a5c-ai/babysitter --skill eval-harness --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
- Define test cases with known-correct outputs - Run agent against each test case - Score: accuracy, completeness, relevance - Compare against baseline performance - Track performance over time ### 2. Skill Quality Testing - Verify skill instructions produce expected outcomes - Test edge cases and boundary conditions - Measure consistency across multiple runs - Check for harmful or incorrect outputs - Validate against ground truth ### 3. Regression Suite - Collection of previously-passing test cases - Run after any agent/skill modification - Flag regressions with before/after comparison - Maintain pass rate threshold (>= 95%) ### 4. Process Verification - End-to-end process execution with known inputs - Verify each phase produces expected outputs - Check task ordering and dependency satisfaction - Measure total execution time ## Quality Scoring ### Accuracy Score (0-100) - Correctness of output vs expected - Partial credit for partially correct outputs - Penalty for hallucinated or fabricated content ### Completeness Score (0-100) - Coverage of required output elements - Missing sections flagged and scored - Bonus for useful additional context ### Consistency Score (0-100) - Run sa
- 2. Skill Quality Testing
- 3. Regression Suite
- 4. Process Verification
- Quality Scoring
- Accuracy Score (0-100)
- Completeness Score (0-100)
- Consistency Score (0-100)
- Composite Score
- When to Use
- Agents Used
What does the eval-harness skill do?
Evaluation harness for testing agent and skill quality through structured benchmarks, regression tests, and quality scoring.
How do I install it?
Run `npx skills add a5c-ai/babysitter --skill eval-harness --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From a5c-ai/babysitter, a repository with 1,642 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.