model-checkpoint-evaluator
When the user needs to evaluate multiple model checkpoints across various benchmarks to identify the best performing model based on evaluation metrics like accuracy, F1-score, or custom scoring functions. This skill scans directories containing model checkpoints, runs benchmark evaluations using predefined scoring algorithms, calculates overall performance scores, and identifies the checkpoint with the highest evaluation metric. It handles complex evaluation workflows involving multiple benchmark categories and mathematical scoring formulas.
npx skills add majiayu000/claude-skill-registry --skill model-checkpoint-evaluator --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Instructions ## Overview This skill evaluates model checkpoints across multiple benchmark categories using predefined scoring algorithms, identifies the best-performing checkpoint based on overall evaluation accuracy, and prepares it for deployment (e.g., uploading to Hugging Face Hub with updated documentation). ## Workflow ### 1. Scan Workspace and Identify Checkpoints - List directories in the workspace to locate checkpoint folders. - Checkpoint folders are typically named with step numbers (e.g., `step_100`, `step_200`). - Extract step numbers from folder names for evaluation. ### 2. Run Benchmark Evaluations - Use the bundled `benchmark_calculator.py` script to compute scores for each checkpoint. - The script implements 15 benchmark scoring functions derived from the original Cython module: - **Core Reasoning Tasks**: Math Reasoning, Logical Reasoning, Common Sense - **Language Understanding**: Reading Comprehension, Question Answering, Text Classification, Sentiment Analysis - **Generation Tasks**: Code Generation, Creative Writing, Dialogue Generation, Summarization - **Specialized Capabilities**: Translation, Knowledge Retrieval, Instruction Following, Safety Evaluation -
- Overview
- Workflow
- 1. Scan Workspace and Identify Checkpoints
- 2. Run Benchmark Evaluations
- 3. Calculate Overall Performance
- 4. Prepare Best Model for Deployment
- Key Decisions
- Notes
What does the model-checkpoint-evaluator skill do?
When the user needs to evaluate multiple model checkpoints across various benchmarks to identify the best performing model based on evaluation metrics like accuracy, F1-score, or custom scoring functions. This skill scans directories containing model checkpoints, runs benchmark evaluations using predefined scoring algorithms, calculates overall performance scores, and identifies the checkpoint with the highest evaluation metric. It handles complex evaluation workflows involving multiple benchmark categories and mathematical scoring formulas.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill model-checkpoint-evaluator --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.