api-ai-promptfoo
Testing and evaluation framework for LLM prompts and applications -- promptfooconfig.yaml, assertions, model-graded evals, red teaming, CI/CD integration, custom providers, and comparative evaluation
npx skills add majiayu000/claude-skill-registry --skill api-ai-promptfoo --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Promptfoo Patterns > **Quick Guide:** Use promptfoo for systematic LLM evaluation. Define prompts, providers, and test cases in `promptfooconfig.yaml`. Use assertion types (`contains`, `is-json`, `llm-rubric`, `similar`, `cost`, `latency`) to validate outputs. Use `promptfoo eval` to run, `promptfoo view` for results UI. Use model-graded assertions (`llm-rubric`, `factuality`) for subjective quality. Use `promptfoo redteam run` for security scanning. Use `--share` flag or `promptfoo share` to share results. All provider API keys come from environment variables -- never hardcode them. --- <critical_requirements> ## CRITICAL: Before Using This Skill > **All code must follow project conventions in CLAUDE.md** (kebab-case, named exports, import ordering, `import type`, named constants) **(You MUST define test cases with explicit `assert` arrays -- tests without assertions only capture output without validating it)** **(You MUST use `llm-rubric` for subjective quality evaluation -- do NOT rely solely on deterministic assertions for natural language output)** **(You MUST set `threshold` on similarity and model-graded assertions -- omitting thresholds uses defaults that may not match yo
- CRITICAL: Before Using This Skill
- Examples Index
- Philosophy
- Core Patterns
- Pattern 1: Basic Configuration
- Pattern 2: Deterministic Assertions
- Pattern 3: Model-Graded Assertions
- Pattern 4: Red Teaming
- Pattern 5: Custom TypeScript Provider
- Pattern 6: CI/CD Integration
- Decision Framework
- Which Assertion Type to Use
- When to Use Red Teaming vs Eval
- Provider Selection
npx promptfoo eval -c promptfooconfig.yaml Always exits 0 -- CI never fails even with broken assertions
What does the api-ai-promptfoo skill do?
Testing and evaluation framework for LLM prompts and applications -- promptfooconfig.yaml, assertions, model-graded evals, red teaming, CI/CD integration, custom providers, and comparative evaluation
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill api-ai-promptfoo --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.