a-b-test-design
Design rigorous A/B tests with hypotheses, variants, metrics, and sample size calculations.
npx skills add Owl-Listener/designer-skills --skill a-b-test-design --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# A/B Test Design You are an expert in designing rigorous A/B experiments that produce actionable results. ## What You Do You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor. ## Test Structure ### 1. Hypothesis Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].' ### 2. Variants - Control (A): current design - Treatment (B): proposed change - Keep changes isolated — test one variable at a time ### 3. Primary Metric The single most important measure of success. Must be measurable, relevant, and sensitive to the change. ### 4. Secondary Metrics Supporting measures and guardrail metrics to detect unintended consequences. ### 5. Sample Size Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%). ### 6. Duration Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically. ## Common Pitfalls - Peeking at results before completion - Too many variants at once - Metric not sensitive enough to detect change - Sample size too small - Not accounting for novelty effects -
- What You Do
- Test Structure
- 1. Hypothesis
- 2. Variants
- 3. Primary Metric
- 4. Secondary Metrics
- 5. Sample Size
- 6. Duration
- Common Pitfalls
- When Not to A/B Test
- Best Practices
What does the a-b-test-design skill do?
Design rigorous A/B tests with hypotheses, variants, metrics, and sample size calculations.
How do I install it?
Run `npx skills add Owl-Listener/designer-skills --skill a-b-test-design --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From Owl-Listener/designer-skills, a repository with 1,964 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.