benchmark-models
Standardized compliance QRA benchmarks against candidate LLMs
npx skills add majiayu000/claude-skill-registry --skill benchmark-models --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# benchmark-models Run standardized compliance QRA tests against candidate LLMs to evaluate accuracy, latency, and cost before deploying to the inference pipeline. ## Usage ### Run a single-model benchmark ```bash ./run.sh run --model deepseek-v3 --suite compliance-basic ``` Runs 20 gold-set compliance QRA questions against the specified model via `/scillm`. Outputs a results table with TEST_CASE, EXPECTED, ACTUAL, MATCH, and LATENCY_MS columns, plus summary metrics (accuracy%, latency_p50, latency_p95, estimated_token_cost). ### Compare multiple models ```bash ./run.sh compare --models "deepseek-v3,llama-3.1-70b" --suite compliance-basic ``` Runs the benchmark for each model and outputs a side-by-side comparison table. ### View last report ```bash ./run.sh report ``` Reads the most recent benchmark results from `~/.embry/benchmark_results.json` and renders a summary. ### Dry run (no LLM calls) ```bash ./run.sh run --model deepseek-v3 --suite compliance-basic --dry-run ./run.sh compare --models "deepseek-v3,llama-3.1-70b" --dry-run ``` Outputs the full benchmark scaffold with 20 test cases and mock results. No LLM calls are made. ## Suites - **compliance-basic**: 20 gold-set QRA qu
- Usage
- Run a single-model benchmark
- Compare multiple models
- View last report
- Dry run (no LLM calls)
- Suites
- Output
What does the benchmark-models skill do?
Standardized compliance QRA benchmarks against candidate LLMs
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill benchmark-models --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.