eval-runner
Run eval scenarios to benchmark Mycelium effectiveness. Execute tasks using reflexion loop, validate against success criteria, record metrics.
npx skills add majiayu000/claude-skill-registry --skill eval-runner --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Eval Runner Benchmark the agent's performance against defined scenarios. Adapted from n-trax eval system. ## Commands ### `run <category/name>` 1. Read YAML from `.claude/evals/scenarios/<category>/<name>.yml` 2. Parse fields (name, category, task_prompt, success_criteria, budget) 3. Execute setup steps if defined 4. Record start time 5. Execute task via reflexion workflow (read corrections first) 6. Record end time and iteration count 7. Validate ALL success criteria 8. Write result JSON to `.claude/evals/results/<timestamp>-<name>.json` 9. Report summary ### `run-all [category]` 1. Glob `.claude/evals/scenarios/**/*.yml` 2. Skip scenarios with `status: retired` 3. For each: run in isolation (git stash), record result, restore 4. Update `.claude/evals/pass-history.json` with each result 5. Aggregate and report ### `run-split <optimization|holdout>` 1. Glob `.claude/evals/scenarios/**/*.yml` 2. Read each YAML, filter by `split` field matching the requested set 3. Skip scenarios with `status: retired` 4. For each matching scenario: run in isolation, record result, restore 5. Update `.claude/evals/pass-history.json` with each result 6. Aggregate and report (label output clearly as
- Commands
- run <category/name>
- run-all [category]
- run-split <optimization|holdout>
- report
- prune
- mine
- Result Format
- Pass History
- Creating Scenarios
What does the eval-runner skill do?
Run eval scenarios to benchmark Mycelium effectiveness. Execute tasks using reflexion loop, validate against success criteria, record metrics.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill eval-runner --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.