run-benchmark
Run an MCP evaluation using mcpbr on SWE-bench or other datasets.
npx skills add majiayu000/claude-skill-registry --skill mcpbr-eval --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Instructions You are an expert at benchmarking AI agents using the `mcpbr` CLI. Your goal is to run valid, reproducible evaluations. ## Critical Constraints (DO NOT IGNORE) 1. **Docker is Mandatory:** Before running ANY `mcpbr` command, you MUST verify Docker is running (`docker ps`). If not, tell the user to start it. 2. **Config is Required:** `mcpbr run` FAILS without a config file. Never guess flags. - IF no config exists: Run `mcpbr init` first to generate a template. - IF config exists: Read it (`cat mcpbr.yaml` or the specified config path) to verify the `mcp_server` command is valid for the user's environment (e.g., check if `npx` or `uvx` is installed). 3. **Workdir Placeholder:** When generating configs, ensure `args` includes `"{workdir}"`. Do not resolve this path yourself; `mcpbr` handles it. 4. **API Key Required:** The `ANTHROPIC_API_KEY` environment variable must be set. Check for it before running evaluations. ## Common Pitfalls to Avoid - **DO NOT** use the `-m` flag unless the user explicitly asks to override the model in the YAML. - **DO NOT** hallucinate dataset names. Valid datasets include: - `SWE-bench/SWE-bench_Lite` (default for SWE-bench) - `SWE-bench/S
- Critical Constraints (DO NOT IGNORE)
- Common Pitfalls to Avoid
- Supported Benchmarks
- Execution Steps
- Example Commands
- Troubleshooting
- Important Reminders
docker ps echo $ANTHROPIC_API_KEY Full evaluation with 5 tasks mcpbr run -c config.yaml -n 5 -v MCP-only evaluation mcpbr run -c config.yaml -M -n 10 Save results and report mcpbr run -c config.yaml -o results.json -r report.md Run CyberGym at level 2 mcpbr run -c config.yaml --benchmark cybergym --level 2 -n 5
What does the run-benchmark skill do?
Run an MCP evaluation using mcpbr on SWE-bench or other datasets.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill mcpbr-eval --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.