ai-eval-plan
Design an evaluation plan for an LLM or AI feature before shipping it. Use when asked how to evaluate a prompt/model/agent, set up an eval harness, define quality metrics for an AI feature, or build a regression gate. Produces an eval plan — task definition, datasets, metrics & rubrics, baselines, automated + human evals, a pass bar, and a regression gate.
npx skills add mohitagw15856/pm-claude-skills --skill ai-eval-plan --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# AI Eval Plan Skill You can't improve an AI feature you can't measure, and "it looks good in the demo" is not measurement. This skill produces an evaluation plan that turns a fuzzy quality goal into a repeatable, gated test — so a prompt change that quietly makes outputs worse can't ship. ## Required Inputs Ask for these only if they aren't already provided: - **The feature & task** — what the model does and what "good output" means to a user. - **Failure modes that matter** — what bad looks like (hallucination, wrong format, unsafe, off-tone, too slow). - **Available data** — any real examples, logs, or labelled cases; or note there are none yet. - **Who judges quality** — automated checks, an LLM judge, human raters, or a mix. - **The decision this gates** — ship/no-ship, model selection, or prompt iteration. ## Output Format ### Eval Plan: [feature] **1. What we're measuring** — the task, and a one-line definition of a good vs. bad response. **2. Eval dataset** - **Cases:** how many, where they come from (real logs > synthetic), and how they're split (smoke set vs. full set). - **Coverage:** the slices/scenarios that must be represented (edge cases, adversarial, each major inpu
- Required Inputs
- Output Format
- Eval Plan: [feature]
- Quality Checks
- Anti-Patterns
- Based On
What does the ai-eval-plan skill do?
Design an evaluation plan for an LLM or AI feature before shipping it. Use when asked how to evaluate a prompt/model/agent, set up an eval harness, define quality metrics for an AI feature, or build a regression gate. Produces an eval plan — task definition, datasets, metrics & rubrics, baselines, automated + human evals, a pass bar, and a regression gate.
How do I install it?
Run `npx skills add mohitagw15856/pm-claude-skills --skill ai-eval-plan --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From mohitagw15856/pm-claude-skills, a repository with 1,255 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.