experiment-readout
Analyse a finished A/B test and write an honest results readout with real statistics. Use when asked to read out an A/B test, analyse experiment results, check if a result is statistically significant, or decide ship/no-ship from test data. Produces a readout — the computed lift, p-value & confidence interval, a significance verdict, guardrail check, and a clear ship / no-ship / iterate recommendation. Includes a stdlib significance calculator.
npx skills add mohitagw15856/pm-claude-skills --skill experiment-readout --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Experiment Readout Skill A test result is only a decision if the statistics are sound — and "variant looks higher" is not a result. This skill computes the lift, the p-value, and a confidence interval from the raw counts, checks the guardrails, and writes an honest readout with a clear ship/no-ship call — flagging the traps (peeking, underpowered, novelty, a significant but tiny effect) that make teams ship noise. ## Required Inputs Ask for these only if they aren't already provided: - **The metric & data** — for a conversion test: users and conversions per variant (control vs. treatment). For a continuous metric: mean, SD, and n per variant. - **The hypothesis** — what you expected and the minimum effect that matters. - **Guardrail metrics** — what shouldn't get worse (revenue, latency, retention). - **Test setup** — planned sample size/duration, and whether it ran to plan (for the peeking check). ## Output Format ### Experiment Readout: [test name] **1. Result** — computed (use the helper): control vs. treatment rate, **absolute & relative lift**, **p-value**, and the **confidence interval** on the difference. | Variant | N | Conversions | Rate | |---|---|---|---| | Control | |
- Required Inputs
- Output Format
- Experiment Readout: [test name]
- Programmatic Helper
- Quality Checks
- Anti-Patterns
- Based On
python3 ab_significance.py <control_n> <control_conv> <treat_n> <treat_conv> python3 scripts/ab_significance.py 10000 800 10000 880 python3 scripts/ab_significance.py 10000 800 10000 880 --json
What does the experiment-readout skill do?
Analyse a finished A/B test and write an honest results readout with real statistics. Use when asked to read out an A/B test, analyse experiment results, check if a result is statistically significant, or decide ship/no-ship from test data. Produces a readout — the computed lift, p-value & confidence interval, a significance verdict, guardrail check, and a clear ship / no-ship / iterate recommendation. Includes a stdlib significance calculator.
How do I install it?
Run `npx skills add mohitagw15856/pm-claude-skills --skill experiment-readout --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From mohitagw15856/pm-claude-skills, a repository with 1,255 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.