building-gold-corpus
Scaffold a synthetic gold-standard annotation project for evaluating OpenMed NER and de-identification models — label schema, annotation guidelines, BRAT or Label Studio config, and disjoint train/dev/test splits. Use when the user wants to create eval fixtures, set up annotation, define a label set, write guidelines, configure an annotation tool, or build a held-out gold set for the OpenMed eval harness. Trigger on \"gold corpus\", \"annotation project\", \"label schema\", \"annotation guidelines\", \"BRAT\", \"Label Studio\", \"train dev test split\", or \"build eval fixtures\" for OpenMed.
npx skills add maziyarpanahi/openmed --skill building-gold-corpus --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Building a Gold Corpus You can't evaluate what you can't measure against. This skill scaffolds a gold-standard annotation project whose output drops straight into the OpenMed eval harness as fixtures. The hard rule: **anything committed to the repo is synthetic**. Licensed clinical corpora (i2b2, n2c2, MIMIC) are DUA-gated — use them at eval time from the user's own copy, never check them in. ## When to use this skill - You need eval fixtures for `benchmarking-clinical-ner` or `evaluating-with-leakage-gates` and have none. - You're standing up an annotation effort: schema, guidelines, tool config. - You need disciplined train/dev/test splits with no leakage between them. - You want a small **synthetic** golden set you can commit and gate on in CI. ## The OpenMed fixture shape (your target output) Annotations must serialize to character-offset spans the harness understands: ```json { "fixtures": [ { "id": "synthetic-0001", "language": "en", "text": "Ms. Jane Roe (MRN 0000000) seen 2099-01-02 for type 2 diabetes.", "gold_spans": [ {"start": 4, "end": 12, "label": "PERSON"}, {"start": 18, "end": 25, "label": "ID_NUM"}, {"start": 32, "end": 42, "label": "DATE"}, {"start": 47, "end":
- When to use this skill
- The OpenMed fixture shape (your target output)
- Quick start — scaffold the project
- Workflow
- Hand-off to / from OpenMed
- Edge cases & gotchas
- Standards & references
What does the building-gold-corpus skill do?
Scaffold a synthetic gold-standard annotation project for evaluating OpenMed NER and de-identification models — label schema, annotation guidelines, BRAT or Label Studio config, and disjoint train/dev/test splits. Use when the user wants to create eval fixtures, set up annotation, define a label set, write guidelines, configure an annotation tool, or build a held-out gold set for the OpenMed eval harness. Trigger on \"gold corpus\", \"annotation project\", \"label schema\", \"annotation guidelines\", \"BRAT\", \"Label Studio\", \"train dev test split\", or \"build eval fixtures\" for OpenMed.
How do I install it?
Run `npx skills add maziyarpanahi/openmed --skill building-gold-corpus --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From maziyarpanahi/openmed, a repository with 4,851 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.