meta-eval
Use when the user wants to build an evaluation system for an LLM/agent application but doesn't know where to start — they have traces, prompts, RAG pipelines, or nothing at all. Also use when the user mentions evaluation, eval, benchmarking, testing LLM quality, measuring agent performance, assessing RAG accuracy, or wants to compare prompts/models. This skill is the entry router: it asks diagnostic questions then recommends which sub-skill (local workflow) to use next.
Profile →npx skills add agentscope-ai/OpenJudge --skill 00-meta-eval --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
<HARD-GATE> NO sub-skill recommendation WITHOUT identifying data_form + label_status (these two pick the entry workflow). ALWAYS give a provisional recommendation once data_form + label_status are known, even if stakes/user_prior are still unknown — then ask the remaining questions to refine the downstream path. Do not withhold the route while waiting on stakes. </HARD-GATE> # Meta Eval Entry router for the eval skill collection. You diagnose what the user has and route them to the right sub-skill. You don't do evaluation yourself — you're the triage desk. Each sub-skill is self-contained: it carries inline the data shapes, statistics, and data principles it needs, so it can be installed and used on its own. ## Checklist You MUST create a task for each item and complete them in order: 1. **Ask 4 diagnostic questions** — data, labels, stakes, domain knowledge 2. **Match triage table** — map user scenario to sub-skill 3. **Recommend sub-skill** — tell the user which workflow to use and why 4. **Record routing decision** — write a brief summary of what was diagnosed and recommended ## Diagnostic Questions Ask these 4 questions (all at once — don't drip-feed): ``` To route you to the r
- Checklist
- Diagnostic Questions
- Triage Table
- Output
- Canonical Workflow (the standard lifecycle)
- Red Flags — STOP and Re-evaluate
- Rationalization Defense
- Common Mistakes
- What This Skill Doesn't Cover
What does the meta-eval skill do?
Use when the user wants to build an evaluation system for an LLM/agent application but doesn't know where to start — they have traces, prompts, RAG pipelines, or nothing at all. Also use when the user mentions evaluation, eval, benchmarking, testing LLM quality, measuring agent performance, assessing RAG accuracy, or wants to compare prompts/models. This skill is the entry router: it asks diagnostic questions then recommends which sub-skill (local workflow) to use next.
How do I install it?
Run `npx skills add agentscope-ai/OpenJudge --skill 00-meta-eval --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From agentscope-ai/OpenJudge, a repository with 764 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.