skillhone-evaluation
Run and interpret skill evaluations. Use when you need to evaluate a skill, run probe/test/PR-val, check if a PR regresses quality, compare two versions, or diagnose why the score dropped. Handles the full eval lifecycle including solver trajectory diagnosis for tool-level error detection.
npx skills add Tencent/SkillHone --skill skillhone-evaluation --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# SkillHone Evaluation Run evaluations and diagnose results to inform improvement decisions. ## Evaluator Harness Model The evaluator is not just a scorer. It is a harness that runs the skill against private tasks and leaves a structured evidence trail: - **Input**: a skill checkout plus an eval repo containing datasets, verifier logic, task contract, and optional compiler/audit helpers. - **Execution**: each eval item runs in an isolated solver workdir. The solver may create files, call scripts, and write the required artifact. - **Verification**: the eval repo verifier scores the produced answer and may call task-local validators, compilers, parsers, renderers, or audit helpers. - **Output JSON**: aggregate score plus redacted per-item trace summaries. - **Workdir evidence**: `trajectory.jsonl`, produced artifacts, and stderr-like signals that explain failures the score cannot explain. This separation matters. A low score may come from weak skill instructions, but it may also come from missing files, tool crashes, invalid compiled artifacts, over-strict verifier rules, or infrastructure errors. Evaluation work is about mapping the failure to the correct harness layer. On Forgejo-
- Evaluator Harness Model
- Core capability: eval.py
- Subagents
- Trajectory Diagnosis
- Compiler Feedback Diagnosis
- Interpreting scores
- Score Provenance
- Splits and data visibility
- Constraints
- Reference
python3 ~/.skillhone/skills/skillhone/scripts/status.py Probe — fast iteration signal python3 ~/.skillhone/skills/skillhone/scripts/eval.py \ Test — final benchmark (NEVER during iteration)
What does the skillhone-evaluation skill do?
Run and interpret skill evaluations. Use when you need to evaluate a skill, run probe/test/PR-val, check if a PR regresses quality, compare two versions, or diagnose why the score dropped. Handles the full eval lifecycle including solver trajectory diagnosis for tool-level error detection.
How do I install it?
Run `npx skills add Tencent/SkillHone --skill skillhone-evaluation --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From Tencent/SkillHone, a repository with 107 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.