product-experiments
Help users design, execute, and analyze product experiments to validate hypotheses and measure true incremental impact while avoiding common statistical pitfalls.
npx skills add RefoundAI/lenny-skills --skill product-experiments --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Product Experimentation Excellence Drive measurable growth and mitigate risk through rigorous A/B testing and data-driven learning. Help the user with product experimentation excellence using insights from 9 guests and posts across Lenny's Podcast and Newsletter. ## How to Help 1. **Hypothesis Definition** - Guide the user in drafting clear, falsifiable hypotheses based on user behavior theories. 2. **Experimental Design** - Help determine the right metrics, sample sizes, and guardrail metrics for a clean test. 3. **Statistical Analysis** - Support the interpretation of p-values, confidence intervals, and potential sample ratio mismatches. 4. **Strategic Evaluation** - Assist in deciding whether to ship, iterate, or kill a feature based on experiment results and long-term business impact. ## Core Principles ### Use long-term holdouts for true incrementality Archie Abrams: "So we constantly will relook at an experiment a year later, see that the way the GMV curve for the distribution was different than we might've originally thought. And that'll actually change what we do from that previous experiment. And so there's a lot of longterm monitoring of experiments over these very long
- How to Help
- Core Principles
- Use long-term holdouts for true incrementality
- Focus experiments on risk mitigation
- Balance success metrics with guardrails
- Normalize a high failure rate
- Skip testing for low-risk best practices
- Apply Twyman's Law to surprising wins
- Templates & Frameworks
- Questions to Help Users
- Common Mistakes to Flag
- Deep Dive
- Related Skills
What does the product-experiments skill do?
Help users design, execute, and analyze product experiments to validate hypotheses and measure true incremental impact while avoiding common statistical pitfalls.
How do I install it?
Run `npx skills add RefoundAI/lenny-skills --skill product-experiments --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From RefoundAI/lenny-skills, a repository with 1,215 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.