Eval Gap Finder

npx skills add majiayu000/claude-skill-registry --skill eval-gap-finder-sunholo-data-ailang-2 --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# Eval Gap Finder Automates the process of finding and closing the gap between Python and AILANG benchmark success rates. Identifies language limitations, prompt gaps, and missing stdlib functions. ## Quick Start **Most common usage:** ```bash # User says: "Find eval gaps" or "Analyze benchmark failures" # This skill will: # 1. Run evals with dev models (gemini-3-flash, claude-haiku-4-5) # 2. Compare Python vs AILANG success rates # 3. Identify benchmarks where Python passes but AILANG fails # 4. Analyze error patterns and categorize them # 5. Check if gaps are documented in prompt # 6. Test proposed examples and add to prompt # 7. Create design docs for language limitations ``` ## When to Use This Skill Invoke this skill when: - User asks to "find eval gaps" or "close the Python-AILANG gap" - User wants to analyze benchmark failures - After running evals and seeing lower AILANG success - User says "why is AILANG failing?" or "improve AILANG benchmarks" - User wants to identify language limitations ## Available Scripts ### `scripts/run_gap_analysis.sh [eval_dir]` Run full gap analysis on eval results. ```bash .claude/skills/eval-gap-finder/scripts/run_gap_analysis.sh eval_results/v

User says: "Find eval gaps" or "Analyze benchmark failures"
This skill will:
ailang eval-suite --models gemini-3-flash,claude-haiku-4-5 --output eval_results/gap-analysis
ailang eval-summary eval_results/gap-analysis
grep -n "pattern" prompts/v0.6.5.md
cat > /tmp/test.ail << 'EOF'
ailang run --caps IO --entry main /tmp/test.ail
ailang eval-suite --models gemini-3-flash,claude-haiku-4-5 --output eval_results/gap-analysis-v2