Eval Analyzer
Analyze evaluation baseline results, identify failure patterns, and generate actionable insights. Use after running eval baselines or when user asks to analyze eval results, check benchmarks, investigate failures, or understand what's failing.
npx skills add majiayu000/claude-skill-registry --skill eval-analyzer --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Eval Analyzer Analyze AILANG evaluation baseline results to identify failure patterns, compare model performance, and generate actionable insights. ## Quick Start **Most common usage:** ```bash # User says: "Analyze the v0.3.24 eval results" # This skill will: # 1. Run eval-analyze to categorize failures (standard eval) # 2. Run agent KPIs to analyze efficiency (agent eval) # 3. Generate summary with jq queries # 4. Identify top failing benchmarks # 5. Show model performance comparison # 6. Provide optimization recommendations ``` **For agent evaluation analysis** (NEW - optimization focus): ```bash # Step 1: Get efficiency metrics (turns, tokens, cost) .claude/skills/eval-analyzer/scripts/agent_kpis.sh eval_results/baselines/v0.3.24 # Step 2: Investigate expensive benchmarks .claude/skills/eval-analyzer/scripts/agent_transcripts.sh eval_results/baselines/v0.3.24 simple_print # Step 3: Compare Python vs AILANG ./tools/compare_agents.sh eval_results/baselines/v0.3.24 ``` **See [`resources/agent_optimization_guide.md`](resources/agent_optimization_guide.md) for complete optimization strategies.** ## When to Use This Skill Invoke this skill when: - User asks to "analyze eval results
- Quick Start
- When to Use This Skill
- Key Eval Commands
- 1. Quick Overview - eval-matrix
- 2. Detailed Analysis - eval-analyze
- 3. Query-Friendly Summary - eval-summary
- 4. Compare Versions - eval-compare
- 5. Fair Comparison (RECOMMENDED) - faircomparison.py
- 6. Validate Results - validateevalresults.py
- Agent Analysis Scripts (NEW!)
- 1. Agent KPIs - Minimize Tokens & Turns
- 2. Agent Transcripts - View AILANG Conversations
- 3. Python vs AILANG Comparison
- Standard Eval Workflow (Non-Agent)
User says: "Analyze the v0.3.24 eval results" This skill will: Step 1: Get efficiency metrics (turns, tokens, cost) Step 2: Investigate expensive benchmarks Step 3: Compare Python vs AILANG ailang eval-matrix eval_results/baselines/v0.3.16 0.3.16 | head -60 Dry run (no design docs, just analysis) ailang eval-analyze -results eval_results/baselines/v0.3.16 -dry-run Full analysis with design doc generation ailang eval-analyze -results eval_results/baselines/v0.3.16
What does the Eval Analyzer skill do?
Analyze evaluation baseline results, identify failure patterns, and generate actionable insights. Use after running eval baselines or when user asks to analyze eval results, check benchmarks, investigate failures, or understand what's failing.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill eval-analyzer --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.