skillhone-synthesis
Use this skill to synthesize closed-form, automatically verifiable benchmark Q/A by exploring a tool environment, building a reusable exploration graph, and mining multiple hard questions from that graph. Use for: building a benchmark, writing eval items, generating evaluation data, closed-form QA, verifiable-answer datasets, synthesising eval data. Applies to any domain with callable tools. Do not use for open-ended writing, subjective scoring, or pure labeling.
npx skills add Tencent/SkillHone --skill skillhone-synthesis --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# SkillHone Synthesis This skill builds a generic closed-form Q/A dataset from real tool use. Closed-form means each question has one intended answer, and the answer can be checked by code: exact match, regex, numeric tolerance, enum, or JSON field checks. The final benchmark should not require a human or an LLM judge for basic grading. The central idea is simple: 1. Explore the target environment with tools. 2. Save what was learned as a graph of entities, relations, and reasoning walks. 3. Mine several Q/A samples from that graph. 4. Validate that each sample is answerable, unique, stable, hard enough, and mechanically gradable. The skill is domain-agnostic. Concrete task files, tool wrappers, and domain examples live outside this skill. ## Task Spec Coverage The task description is the source of truth for both the solver prompt and the verifier. When a task spec uses words such as "must", "must not", "required", "forbidden", "output format", "validation", "quality", or "acceptance", treat those clauses as benchmark requirements. For every requirement in the task spec that can be checked mechanically, the generated verifier should include a corresponding `scores` key. This includ
- Task Spec Coverage
- Inputs
- Simple, Hard, And Broken Questions
- Pipeline
- Exploration Graph
- Difficulty Construction
- Output
- Verification Rules
- What This Skill Is Not
What does the skillhone-synthesis skill do?
Use this skill to synthesize closed-form, automatically verifiable benchmark Q/A by exploring a tool environment, building a reusable exploration graph, and mining multiple hard questions from that graph. Use for: building a benchmark, writing eval items, generating evaluation data, closed-form QA, verifiable-answer datasets, synthesising eval data. Applies to any domain with callable tools. Do not use for open-ended writing, subjective scoring, or pure labeling.
How do I install it?
Run `npx skills add Tencent/SkillHone --skill skillhone-synthesis --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From Tencent/SkillHone, a repository with 107 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.