agentv-eval-analyzer
Analyze evaluation results for quality improvements — identify LLM-judge evaluators replaceable with deterministic assertions, flag weak/vague assertions, and surface cost/quality opportunities. Use after running evals to improve your evaluation config.
npx skills add majiayu000/claude-skill-registry --skill agentv-eval-analyzer --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# AgentV Eval Analyzer Analyze JSONL evaluation results and produce actionable suggestions for improving eval quality, reducing cost, and increasing reliability. ## When to Use - After running `agentv eval` and wanting to improve your evaluation config - When evaluations are slow or expensive and you suspect LLM-judges are doing deterministic work - When reviewing eval quality before sharing or publishing evaluation files - When triaging flaky evaluations that produce inconsistent scores ## Quick Start ```bash # Find your most recent results file agentv trace list --limit 5 # Dispatch the eval-analyzer agent with the results file # Agent: eval-analyzer # Parameters: # results-file: <path-to-results.jsonl> # eval-path: <path-to-eval.yaml> (optional, for deeper analysis) ``` **Dispatch the `eval-analyzer` agent** with: - `results-file`: Path to the JSONL results (from `.agentv/results/` or `agentv trace list`) - `eval-path` (optional): Path to the EVAL.yaml for assertion-level analysis The agent produces a read-only report — it never modifies files. ## What It Detects ### 1. Deterministic-Upgrade Candidates (highest value) LLM-judge evaluators doing work that a deterministic assertio
- When to Use
- Quick Start
- What It Detects
- 1. Deterministic-Upgrade Candidates (highest value)
- 2. Weak Assertion Detection
- 3. Cost/Quality Flags
- 4. Multi-Provider Variance
- Applying Suggestions
- Limitations
Find your most recent results file agentv trace list --limit 5 Dispatch the eval-analyzer agent with the results file
What does the agentv-eval-analyzer skill do?
Analyze evaluation results for quality improvements — identify LLM-judge evaluators replaceable with deterministic assertions, flag weak/vague assertions, and surface cost/quality opportunities. Use after running evals to improve your evaluation config.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill agentv-eval-analyzer --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.