swe-bench-lite

npx skills add majiayu000/claude-skill-registry --skill benchmark-swe-lite --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# Instructions This skill provides a streamlined way to run the SWE-bench Lite benchmark with pre-configured defaults. ## What This Skill Does This skill runs a quick SWE-bench Lite evaluation with: - 5 sample tasks (configurable) - Verbose output for visibility - Results saved to `results.json` - Report saved to `report.md` ## Prerequisites Check Before running, verify: 1. **Docker is running:** ```bash docker ps ``` 2. **API key is set:** ```bash echo $ANTHROPIC_API_KEY ``` 3. **Config file exists:** - Check for `mcpbr.yaml` in the current directory - If missing, run `mcpbr init` to generate it ## Default Command The default command for SWE-bench Lite: ```bash mcpbr run -c mcpbr.yaml --dataset SWE-bench/SWE-bench_Lite -n 5 -v -o results.json -r report.md ``` ## Customization Options Users can customize the run by modifying: - **Sample size:** Change `-n 5` to any number (or remove for full dataset) - **Config file:** Change `-c mcpbr.yaml` to point to a different config - **Verbosity:** Use `-vv` for very verbose output - **Output files:** Change `results.json` and `report.md` to different paths ## Example Variations ### Minimal quick test (1 task) ```bash mcpbr run -c mcpbr.yaml

docker ps
echo $ANTHROPIC_API_KEY
mcpbr run -c mcpbr.yaml --dataset SWE-bench/SWE-bench_Lite -n 5 -v -o results.json -r report.md
mcpbr run -c mcpbr.yaml -n 1 -v
mcpbr run -c mcpbr.yaml --dataset SWE-bench/SWE-bench_Lite -v -o results.json
mcpbr run -c mcpbr.yaml -n 5 -M -v -o results.json
mcpbr run -c mcpbr.yaml -t astropy__astropy-12907 -t django__django-11099 -v