ab-test-setup
When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
Profile →npx skills add alirezarezvani/claude-skills --skill ab-test-setup --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# A/B Test Setup You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results. ## Initial Assessment **Check for product marketing context first:** If `.claude/product-marketing-context.md` exists, read it before asking questions. Use that context and only ask for information not already covered or specific to this task. Before designing a test, understand: 1. **Test Context** - What are you trying to improve? What change are you considering? 2. **Current State** - Baseline conversion rate? Current traffic volume? 3. **Constraints** - Technical complexity? Timeline? Tools available? --- ## Core Principles ### 1. Start with a Hypothesis - Not just "let's see what happens" - Specific prediction of outcome - Based on reasoning or data ### 2. Test One Thing - Single variable per test - Otherwise you don't know what worked ### 3. Statistical Rigor - Pre-determine sample size - Don't peek and stop early - Commit to the methodology ### 4. Measure What Matters - Primary metric tied to business value - Secondary metrics for context - Guardrail metrics to prevent harm --- ## Hypothesis Framework ### Structure ```
- Initial Assessment
- Core Principles
- 1. Start with a Hypothesis
- 2. Test One Thing
- 3. Statistical Rigor
- 4. Measure What Matters
- Hypothesis Framework
- Structure
- Example
- Test Types
- Sample Size
- Calculate It (bundled tool)
- Quick Reference
- Metrics Selection
python3 scripts/sample_size_calculator.py --baseline 0.05 --mde 0.20 # human-readable python3 scripts/sample_size_calculator.py --baseline 0.05 --mde 0.20 --json # for pipelines python3 scripts/sample_size_calculator.py --baseline 0.05 --mde 0.20 --daily-traffic 2000 # adds test-duration estimate
What does the ab-test-setup skill do?
When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
How do I install it?
Run `npx skills add alirezarezvani/claude-skills --skill ab-test-setup --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From alirezarezvani/claude-skills, a repository with 23,791 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.