prompt-optimizer
A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results.
npx skills add majiayu000/claude-skill-registry --skill prompt-optimizer-haabe-mycelium --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Prompt Optimizer Systematically improve Mycelium instructions through measurement. Adapted from n-trax. ## Commands ### `baseline` -- Capture current performance 1. Run `/eval-runner run-split optimization` — record as optimization scores 2. Run `/eval-runner run-split holdout` — record as holdout scores 3. Record both to `.claude/optimization/baseline.json`: timestamp, CLAUDE.md hash, optimization metrics, holdout metrics, overall and per-category metrics ### `test <variant>` -- Test a variant 1. Read variant from `.claude/optimization/variants/<variant>.md` 2. Apply the CLAUDE.md changes described 3. Run `/eval-runner run-split optimization` — this is the hill-climbing signal 4. Run `/eval-runner run-split holdout` — this validates generalization 5. Store results in `.claude/optimization/results/<variant>.json` 6. Compare against baseline. Flag **overfitting** if optimization improves but holdout degrades. 7. Do NOT auto-revert -- let user decide ### `report` -- Compare all variants Generate comparison table with split-aware columns: ``` | Variant | Opt Pass Rate | Holdout Pass Rate | Delta Opt | Delta Holdout | Overfit? | Decision | ``` Flag `Overfit? = YES` when optimization
- Commands
- baseline -- Capture current performance
- test <variant> -- Test a variant
- report -- Compare all variants
- exemplar <eval-name> -- Capture winning trajectory
- Workflow
What does the prompt-optimizer skill do?
A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill prompt-optimizer-haabe-mycelium --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.