auditing-subgroup-fairness
Audit an OpenMed NER or de-identification model for performance disparities across demographic subgroups (sex, age band, race/ethnicity when available) using openmed.eval.fairness_report. Use when the user wants per-subgroup recall and leakage, wants to check whether de-identification under-protects a group, wants to surface a documentation gap where subgroup data is missing, or needs equalized-odds-style disparity numbers for a clinical model. Trigger on \"fairness\", \"subgroup\", \"bias audit\", \"disparity\", \"equalized odds\", \"under-protected group\", \"per-group recall\", or \"STANDIN
npx skills add maziyarpanahi/openmed --skill auditing-subgroup-fairness --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Auditing Subgroup Fairness An aggregate pass can hide a group the model fails. For de-identification that failure has a name: **under-protection** — PHI that leaks more often for one demographic group than another. `openmed.eval.fairness_report` slices leakage and recall by gold-span group so disparities surface before deployment, not after a breach. ## When to use this skill - You want per-subgroup recall and leakage for a de-id or NER model. - You suspect (or must rule out) that one group is under-protected. - You need disparity numbers for a clinical AI governance review. - You need to document *which* subgroups you couldn't evaluate (the data gap). ## What it measures For each surrogate group `fairness_report` returns: - **leakage_rate** — fraction of that group's gold PHI characters left exposed (the de-id harm metric). - **recall** — fraction of that group's gold spans detected. - **leakage_disparity** — `max - min` leakage across groups (the gap to close). - **worst_group** / **worst_group_leakage** — the most-failed group. Group membership comes from a `group` tag in each gold span's `metadata` (keys `group`, `demographic_group`, or `surrogate_group`); ungrouped spans fal
- When to use this skill
- What it measures
- Quick start
- Workflow
- Hand-off to / from OpenMed
- Edge cases & gotchas
- Standards & references
What does the auditing-subgroup-fairness skill do?
Audit an OpenMed NER or de-identification model for performance disparities across demographic subgroups (sex, age band, race/ethnicity when available) using openmed.eval.fairness_report. Use when the user wants per-subgroup recall and leakage, wants to check whether de-identification under-protects a group, wants to surface a documentation gap where subgroup data is missing, or needs equalized-odds-style disparity numbers for a clinical model. Trigger on \"fairness\", \"subgroup\", \"bias audit\", \"disparity\", \"equalized odds\", \"under-protected group\", \"per-group recall\", or \"STANDIN
How do I install it?
Run `npx skills add maziyarpanahi/openmed --skill auditing-subgroup-fairness --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From maziyarpanahi/openmed, a repository with 4,851 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.