ab-test-setup
Structured guide for setting up A/B tests with mandatory gates for hypothesis, metrics, and execution readiness.
Profile →npx skills add sickn33/agentic-awesome-skills --skill ab-test-setup --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# A/B Test Setup ## 1️⃣ Purpose & Scope Ensure every A/B test is **valid, rigorous, and safe** before a single line of code is written. - Prevents "peeking" - Enforces statistical power - Blocks invalid hypotheses --- ## 2️⃣ Pre-Requisites You must have: - A clear user problem - Access to an analytics source - Roughly estimated traffic volume ### Hypothesis Quality Checklist A valid hypothesis includes: - Observation or evidence - Single, specific change - Directional expectation - Defined audience - Measurable success criteria --- ## 3️⃣ Hypothesis Lock (Hard Gate) Before designing variants or metrics, you MUST: - Present the **final hypothesis** - Specify: - Target audience - Primary metric - Expected direction of effect - Minimum Detectable Effect (MDE) Ask explicitly: > “Is this the final hypothesis we are committing to for this test?” **Do NOT proceed until confirmed.** --- ## 4️⃣ Assumptions & Validity Check (Mandatory) Explicitly list assumptions about: - Traffic stability - User independence - Metric reliability - Randomization quality - External factors (seasonality, campaigns, releases) If assumptions are weak or violated: - Warn the user - Recommend delaying or redesigni
- 1️⃣ Purpose & Scope
- 2️⃣ Pre-Requisites
- Hypothesis Quality Checklist
- 3️⃣ Hypothesis Lock (Hard Gate)
- 4️⃣ Assumptions & Validity Check (Mandatory)
- 5️⃣ Test Type Selection
- 6️⃣ Metrics Definition
- 7️⃣ Sample Size & Duration
- Tracking Verification (Required before Gate 8)
- 8️⃣ Execution Readiness Gate (Hard Stop)
- Running the Test
- During the Test
- Analyzing Results
- Analysis Discipline
What does the ab-test-setup skill do?
Structured guide for setting up A/B tests with mandatory gates for hypothesis, metrics, and execution readiness.
How do I install it?
Run `npx skills add sickn33/agentic-awesome-skills --skill ab-test-setup --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From sickn33/agentic-awesome-skills, a repository with 44,414 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.