model-ab-test
Use when validating model downgrades for skills or agents. Runs A/B comparison between current and proposed model, scores outputs against test criteria, produces verdict.
npx skills add majiayu000/claude-skill-registry --skill model-ab-test --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Model A/B Test Harness Run each test case from a definition file: dispatch two subagents (baseline vs proposed model) in parallel, score outputs binary per criterion, write evidence files, produce verdict. ## Phase 1: Parse Test Definitions Split `$ARGUMENTS` — first token = file path, second (optional) = test name filter (case-insensitive substring match on `### <name>` headings). Read the file. For each test case block, extract: | Field | Source | |-------|--------| | `name` | `### <name>` heading (before parenthetical) | | `proposed_model` | From `(Opus -> X)` — extract target as lowercase: `sonnet` or `haiku` | | `test_input` | Content under `**Test input:**` until next bold section | | `green_criteria` | List under `**GREEN criteria**` | | `red_indicators` | List under `**RED indicators**` | Skip cases with missing fields (log warning). Skip `~~strikethrough~~` or `EXEMPTED` headings entirely. ## Phase 2: Locate Content + Preloaded Skills For each test case `name`: 1. Glob `plugins/*/skills/{name}/SKILL.md` — if found, this is `primary_content` (type: skill) 2. Else glob `plugins/*/agents/{name}.md` — if found, this is `primary_content` (type: agent) 3. If agent has `skills:
- Phase 1: Parse Test Definitions
- Phase 2: Locate Content + Preloaded Skills
- Phase 3: Dispatch A/B Subagents
- Phase 4: Score Results
- Phase 5: Summary Report
- Sequencing
- Discipline Rules
- Common Mistakes
What does the model-ab-test skill do?
Use when validating model downgrades for skills or agents. Runs A/B comparison between current and proposed model, scores outputs against test criteria, produces verdict.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill model-ab-test --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.