benchmark-fetcher
Fetch benchmark performance data from 6 leaderboard websites using Playwright MCP and update model manifests with the latest scores. Supports SWE-bench, TerminalBench, SciCode, LiveCodeBench, MMMU, MMMU Pro, and WebDevArena benchmarks.
npx skills add majiayu000/claude-skill-registry --skill benchmark-fetcher --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Benchmark Fetcher Skill Automate the fetching of benchmark performance data from leaderboard websites and update model manifests with the latest scores using advanced browser automation. ## Overview This skill extends benchmark data collection by automating visits to 6 major AI model leaderboard websites, extracting performance scores, and updating model manifests in `manifests/models/` with the latest benchmark data. **Key Features:** - **Automated Data Collection**: Uses Playwright MCP to visit and extract data from 6 leaderboard websites - **Intelligent Model Mapping**: Maps website model names to manifest IDs using configurable mappings - **Always Overwrite**: Updates manifests with latest benchmark values - **Error Resilient**: Retry logic with exponential backoff and graceful degradation - **Comprehensive Reporting**: Detailed completion reports with unmapped models and update statistics ## Supported Benchmarks | Benchmark | Website | Manifest Field | Format | |-----------|---------|----------------|--------| | **SWE-bench** | https://www.swebench.com | `sweBench` | Percentage (0-100) | | **TerminalBench** | https://www.tbench.ai/leaderboard/terminal-bench/2.0 | `terminalBe
- Overview
- Supported Benchmarks
- Usage
- Fetch All Benchmarks
- Fetch Specific Benchmarks
- Fetch for Specific Models
- Dry Run Mode
- Model Name Mapping
- How Mapping Works
- Mapping Strategy
- Adding New Mappings
- Data Extraction Process
- High-Level Workflow
- Website-Specific Extractors
node .claude/skills/benchmark-fetcher/scripts/fetch-benchmarks.mjs Fetch only SWE-bench and TerminalBench node .claude/skills/benchmark-fetcher/scripts/fetch-benchmarks.mjs --benchmarks swebench,terminalBench Fetch only LiveCodeBench node .claude/skills/benchmark-fetcher/scripts/fetch-benchmarks.mjs --benchmarks liveCodeBench Update only Claude Sonnet 4.5 and GPT-4o node .claude/skills/benchmark-fetcher/scripts/fetch-benchmarks.mjs --models claude-sonnet-4-5,gpt-4o node .claude/skills/benchmark-fetcher/scripts/fetch-benchmarks.mjs --dry-run Run schema validation npm run test:validate
What does the benchmark-fetcher skill do?
Fetch benchmark performance data from 6 leaderboard websites using Playwright MCP and update model manifests with the latest scores. Supports SWE-bench, TerminalBench, SciCode, LiveCodeBench, MMMU, MMMU Pro, and WebDevArena benchmarks.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill benchmark-fetcher --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.