19-ab-test-setup-global
Design valid A/B tests for global marketing — hypothesis formulation, sample size calculation, statistical significance, multi-arm testing, primary vs secondary metrics. Tools: Optimizely, VWO, Google Optimize (sunset 2023, alternatives), built-in platform tests (Meta, Google). Trigger: 'A/B test', 'split test', 'multivariate test', 'experiment design', 'statistical significance', 'sample size calculator'.
Profile →npx skills add minhnv0807/ai-business-skills --skill 19-ab-test-setup-global --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# A/B Test Setup (Global) > Run experiments that produce decisions, not noise. Most "A/B tests" in marketing are underpowered, peeked-at, and badly hypothesized — meaning the team learns nothing and ships the louder variant. --- ## For Newbies A valid A/B test answers one question: "Did this change cause a real improvement, or am I seeing noise?" To answer it credibly you need four things: 1. A **specific hypothesis** with a numeric prediction 2. **One variable changed** (everything else identical) 3. **Enough sample** to detect the effect you care about 4. **Statistical significance** before you call a winner (typically p < 0.05) If any one of these is missing, you don't have an A/B test — you have a coin flip with extra steps. **Common newbie mistake:** running a test for 3 days, seeing variant B 40% higher, declaring victory, and shipping. Three days is too short to absorb day-of-week effects, and small samples produce wild swings. Variant B may revert (or reverse) by day 14. --- ## Step 0 — Read Context Read `.agents/product-marketing-context.md` if it exists. Audience size, average traffic, and current conversion rate determine whether a test is even feasible. --- ## Step 1 —
- For Newbies
- Step 0 — Read Context
- Step 1 — Information Gathering
- The 7 Principles of a Valid A/B Test
- 1. Test exactly one variable
- 2. Hypothesize with a number
- 3. Sufficient sample size
- 4. Sufficient duration
- 5. Don't peek
- 6. Statistical significance: p < 0.05
- 7. Document everything
- Sample Size Calculation
- Quick formula
- Worked example A — landing page CRO
What does the 19-ab-test-setup-global skill do?
Design valid A/B tests for global marketing — hypothesis formulation, sample size calculation, statistical significance, multi-arm testing, primary vs secondary metrics. Tools: Optimizely, VWO, Google Optimize (sunset 2023, alternatives), built-in platform tests (Meta, Google). Trigger: 'A/B test', 'split test', 'multivariate test', 'experiment design', 'statistical significance', 'sample size calculator'.
How do I install it?
Run `npx skills add minhnv0807/ai-business-skills --skill 19-ab-test-setup-global --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From minhnv0807/ai-business-skills, a repository with 521 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.