eval-harness

npx skills add mturac/everything-openai-codex --skill eval-harness --agent codex

Same command for any agent — swap --agent for claude-code, cursor, copilot.

# Eval Harness Skill A formal evaluation framework for OpenAI Codex sessions, implementing eval-driven development (EDD) principles. ## When to Activate - Setting up eval-driven development (EDD) for AI-assisted workflows - Defining pass/fail criteria for OpenAI Codex task completion - Measuring agent reliability with pass@k metrics - Creating regression test suites for prompt or agent changes - Benchmarking agent performance across model versions ## Philosophy Eval-Driven Development treats evals as the "unit tests of AI development": - Define expected behavior BEFORE implementation - Run evals continuously during development - Track regressions with each change - Use pass@k metrics for reliability measurement ## Eval Types ### Capability Evals Test if Codex can do something it couldn't before: ```markdown [CAPABILITY EVAL: feature-name] Task: Description of what Codex should accomplish Success Criteria: - [ ] Criterion 1 - [ ] Criterion 2 - [ ] Criterion 3 Expected Output: Description of expected result ``` ### Regression Evals Ensure changes don't break existing functionality: ```markdown [REGRESSION EVAL: feature-name] Baseline: SHA or checkpoint name Tests: - existing-test-1:

Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
Run capability evals
Run regression evals
npm test -- --testPathPattern="existing"
Generate report