spec-driven-eval
Scores how completely an implementation fulfills a PRD/spec, case by case, and produces a single comparable final grade. Invoke only when explicitly named (e.g. run spec-driven-eval); do not auto-trigger. Use when benchmarking spec-driven implementations, grading acceptance criteria, evaluating whether a feature was 100% implemented, comparing multiple implementations of the same PRD, or auditing implementation and test coverage (unit and e2e) against product requirements. Do NOT use for planning or building features (use tlc-spec-driven), writing PRDs, or general code review unrelated to a sp
npx skills add tech-leads-club/agent-skills --skill spec-driven-eval --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
What it does
Evaluates a spec-driven-development effort against a PRD, scoring each acceptance criterion (AC) for both implementation and tests separately, then rolls up to a single final grade. Designed for benchmarking across multiple implementations and for auditing coverage of requirements and tests (unit and e2e). The evaluation is binary (MET/UNMET) per atomic check, with a diff-surface evidence approach and a reproducible scoring baseline.
How it works
- Treats two independent subjects: (1) framework respect and requirement extraction (E and S axes) and (2) harness implementation and test completeness (T and G axes).
- Uses a binary checklist per AC: I-checks for observable behaviors in production code and T-checks for verification at unit and/or e2e levels.
- Enforces evidence rules: MET checks must cite
file:lineorfile:startLine-endLine. If no evidence is provided, the check is UNMET. - Requires documentation of the diff surface (changed files) as part of the scoring context.
- Uses rules for payload correctness: for ACs that mention specific fields or entities in emitted data, open the actual payload and verify presence of each named field as distinct I-checks.
- Distinguishes product-controlled choices with multiple paths, assigning separate I-checks for each path when applicable.
- Includes a wiring I-check if asynchronous inbound events affect observable side-effects, ensuring events reach and dispatch to the correct handler.
- Computes the final grade via a scripted aggregation that consumes per-AC I/T fractions and a priority-weight table, producing a two-decimal Final score.
When to use
- When asked to evaluate/score this PRD case by case and provide a final grade.
- To determine whether a feature was implemented 100% according to the PRD.
- To benchmark multiple spec-driven implementations of the same PRD.
- To audit implementation and test coverage (unit + e2e) against product requirements.
What it can touch
- The evaluation relies on evidence from production code and tests, identified via
file:linemarkers. It requires access to the implementation code, unit tests, e2e tests, and derived spec artifacts (spec.md,tasks.md).
Caveats
- The evaluator is read-only over the subject; no code changes are allowed during scoring.
- The framework performs three independent passes (k=3) and uses majority verdicts per check before computing final numbers.
- The scoring relies on a frozen checklist per PRD; changes to the AC baseline should be reflected in the baseline report and not in ongoing scoring.
# Spec-Driven Implementation Evaluation Evaluate a spec-driven-development (SDD) effort against a PRD **case by case** (acceptance criterion by acceptance criterion), scoring **implementation** and **tests** separately, and roll up to a single comparable final grade. Designed for benchmarking: the same PRD evaluated across different SDD frameworks — and the same effort evaluated twice — must yield comparable, reproducible numbers. ### Two subjects, two questions This eval answers two independent questions, and keeps their verdicts separate because they fail independently: 1. **How good is the framework at respecting and extracting requirements?** — Did it honor the PRD and stay in bounds (*respect*: `Final` implementation side + Scope `S`), and did it surface the implicit requirements the PRD only implied, without noise (*extract*: Elicitation `E`)? 2. **How good is the harness at ensuring they are all implemented?** — Does the test suite prove every sanctioned requirement is actually built (`T` + Engineering Gates `G`)? > **The linkage that makes "all implemented" well-defined:** the harness is held accountable for the **full sanctioned requirement set = `PRD acceptance criteria ∪
- Two subjects, two questions
- When to use
- Inputs required
- Quick start
- Core rules (read first)
- Reproducibility rules
- Scoring model
- Unit of scoring: the acceptance criterion (AC), decomposed into binary checks
- Per-AC, per-story, final
- Grade bands
- Bias controls
- Reported beside the grade (NOT folded into Final — keep comparable)
- Test distribution by tier D (reported beside)
- Erecall — did it find what it should have?
git diff <base>..<head> --name-only # branch or PR git diff --name-only HEAD # uncommitted changes git status --short # include untracked new files git diff <base>..<head> --name-only # for a branch/PR git diff --name-only HEAD~1 # for the last commit git diff --name-only HEAD # for uncommitted changes git status --short # for untracked new files
What does the spec-driven-eval skill do?
Scores how completely an implementation fulfills a PRD/spec, case by case, and produces a single comparable final grade. Invoke only when explicitly named (e.g. run spec-driven-eval); do not auto-trigger. Use when benchmarking spec-driven implementations, grading acceptance criteria, evaluating whether a feature was 100% implemented, comparing multiple implementations of the same PRD, or auditing implementation and test coverage (unit and e2e) against product requirements. Do NOT use for planning or building features (use tlc-spec-driven), writing PRDs, or general code review unrelated to a sp
How do I install it?
Run `npx skills add tech-leads-club/agent-skills --skill spec-driven-eval --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From tech-leads-club/agent-skills, a repository with 4,983 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.