benchmarking-clinical-ner
Score an OpenMed clinical or biomedical NER model against a user-supplied gold corpus with entity-level precision, recall, and F1, then break errors down per label. Use when the user wants a seqeval-style scorecard, strict vs partial (relaxed) span matching, a per-label confusion matrix, false-negative / false-positive examples, or to debug why a model misses entities. Trigger on \"evaluate NER\", \"entity-level F1\", \"seqeval\", \"precision recall F1\", \"confusion matrix\", \"error analysis\", \"strict vs partial match\", or \"score against gold\" in an OpenMed context. The gold corpus is u
npx skills add maziyarpanahi/openmed --skill benchmarking-clinical-ner --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Benchmarking Clinical NER This skill produces an honest entity-level scorecard for an OpenMed NER model: precision / recall / F1 plus a per-label error breakdown. It scores **spans**, not tokens, because clinical entities are multi-token ("type 2 diabetes mellitus") and token-level accuracy hides boundary errors. Reported numbers are **entity-level** in the seqeval tradition (CoNLL-2000 / SemEval-2013 families). ## When to use this skill - You have a gold-annotated clinical corpus and an OpenMed NER model to score. - You want strict (exact-boundary) and partial (relaxed-overlap) span F1. - You need per-label numbers, not one aggregate — DRUG recall ≠ DISEASE recall. - You need to *explain* the errors: what was missed, what was spurious, what was mislabeled. For PHI de-id specifically, gate on leakage with `evaluating-with-leakage-gates` instead of (or in addition to) F1. ## Match modes | Mode | Counts a hit when… | Use for | | --- | --- | --- | | **Strict / exact** | predicted span boundaries **and** label match gold exactly | release scoring, boundary-sensitive tasks | | **Partial / relaxed** | predicted span **overlaps** gold with the right label | recall-oriented triage, token
- When to use this skill
- Match modes
- Quick start
- Workflow
- Hand-off to / from OpenMed
- Edge cases & gotchas
- Standards & references
What does the benchmarking-clinical-ner skill do?
Score an OpenMed clinical or biomedical NER model against a user-supplied gold corpus with entity-level precision, recall, and F1, then break errors down per label. Use when the user wants a seqeval-style scorecard, strict vs partial (relaxed) span matching, a per-label confusion matrix, false-negative / false-positive examples, or to debug why a model misses entities. Trigger on \"evaluate NER\", \"entity-level F1\", \"seqeval\", \"precision recall F1\", \"confusion matrix\", \"error analysis\", \"strict vs partial match\", or \"score against gold\" in an OpenMed context. The gold corpus is u
How do I install it?
Run `npx skills add maziyarpanahi/openmed --skill benchmarking-clinical-ner --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From maziyarpanahi/openmed, a repository with 4,851 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.