exps-performance
Documentation for the main LLM performance experiments (exps_performance)
npx skills add majiayu000/claude-skill-registry --skill exps-performance --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# exps_performance - Main Performance Experiments ## Purpose Run LLM inference on algorithmic problems, collect CoT rationales and evaluate correctness. ## Key Files | File | Purpose | |------|---------| | `src/exps_performance/main.py` | Main runner | | `src/exps_performance/dataset.py` | Dataset definitions | | `src/exps_performance/arms.py` | Evaluation arms (NL, code_sim, code_exec) | | `src/exps_performance/llm.py` | LLM interface | | `src/exps_performance/analysis.py` | Analysis and plotting | | `src/exps_performance/noise.py` | Noise injection experiments | ## Results Structure ``` src/exps_performance/results/ {model}_seed{seed}/ tb/ run_{timestamp}/ res.jsonl # Main results file ``` ## res.jsonl Schema | Field | Description | |-------|-------------| | `kind` | Problem type (e.g., "knap", "bellman_ford") | | `digit` | Problem size/digits | | `nl_correct` | NL evaluation result | | `code_correct` | Code execution result | | `sim_correct` | Code similarity result | | `model` | Model name | | `seed` | Random seed | ## Data Sources - Performance results: `src/exps_performance/results/{model}_seed{seed}/tb/run_*/res.jsonl` - Excludes `gsm8k` from accuracy calculations in logisti
- Purpose
- Key Files
- Results Structure
- res.jsonl Schema
- Data Sources
- Running Commands
Run main experiment uv run python src/exps_performance/main.py --model <model> --seed <seed> Run analysis uv run python src/exps_performance/analysis.py
What does the exps-performance skill do?
Documentation for the main LLM performance experiments (exps_performance)
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill exps-performance --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.