promptfoo
Promptfoo evaluation framework for testing and comparing LLM outputs. Use when writing eval configs, creating test cases, debugging eval runs, or working with assertions.
npx skills add majiayu000/claude-skill-registry --skill promptfoo --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Promptfoo [Promptfoo](https://www.promptfoo.dev/) is a CLI tool for testing and comparing LLM outputs. ## Config File The CLI auto-discovers `promptfooconfig.yaml` in the current directory. Use `-c path` for other locations. Supported extensions: `.yaml`, `.json`, `.js` ## Configuration ```yaml # yaml-language-server: $schema=https://promptfoo.dev/config-schema.json description: "What this eval tests" prompts: - file://prompt.txt - | Inline prompt with {{variable}} substitution providers: - anthropic:messages:claude-sonnet-4-5-20250929 defaultTest: options: provider: config: temperature: 0.0 max_tokens: 4096 tests: - description: "What this case tests" vars: variable: "value" from_file: file://data/input.txt assert: - type: contains value: "expected substring" # Or load tests from files tests: file://cases/all.yaml outputPath: ./results.json evaluateOptions: maxConcurrency: 4 ``` ## Provider IDs | Model | ID | |-------|----| | Opus 4.5 | `anthropic:messages:claude-opus-4-5-20251101` | | Sonnet 4.5 | `anthropic:messages:claude-sonnet-4-5-20250929` | | Haiku 4.5 | `anthropic:messages:claude-haiku-4-5-20251001` | Provider config: `temperature`, `max_tokens`, `top_p`, `top_k`, `tools
- Config File
- Configuration
- Provider IDs
- Prompts
- Assertion Types
- llm-rubric
- javascript
- Test Organization
- CLI
- References
npx promptfoo eval # Run with auto-discovered config npx promptfoo eval -c path/to/config.yaml # Specific config npx promptfoo eval --filter-metadata key=v # Filter tests npx promptfoo view # Web UI for results npx promptfoo cache clear # Clear result cache
What does the promptfoo skill do?
Promptfoo evaluation framework for testing and comparing LLM outputs. Use when writing eval configs, creating test cases, debugging eval runs, or working with assertions.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill promptfoo --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.