gating-deid-leakage
Add a CI gate that fails the build when an OpenMed de-identification model's recall on a held-out PHI set drops below threshold or any critical identifier leaks. Use when the user wants a pytest test or CLI step that exits nonzero on de-id regression, wants to wire OpenMed's leakage-first release gates into GitHub Actions / CI, needs a recall floor plus zero-leakage assertion against a synthetic held-out set, or wants to block merges that weaken de-identification. Trigger on \"CI gate\", \"fail the build\", \"regression test\", \"de-id recall threshold\", \"block the merge\", \"exit nonzero\",
npx skills add maziyarpanahi/openmed --skill gating-deid-leakage --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Gating De-id Leakage in CI Logs, baselines, and models drift. The only durable defense is a gate that runs on every change and **fails closed** when de-identification regresses. This skill operationalizes OpenMed's leakage-first ethos into a CI check: recall must stay above the floor and **critical leakage must be exactly zero**, or the build goes red. ## When to use this skill - You want a pytest test or CLI step that exits nonzero on de-id regression. - You need to block PRs that drop PHI recall or introduce a leak. - You want OpenMed's release gates (`ReleaseGate`, G1a–G8) enforced in CI. - You maintain a synthetic held-out PHI set and want it checked automatically. For the full gate semantics see `evaluating-with-leakage-gates`; this skill is about *wiring it into CI so it fails the build*. ## Quick start — a pytest gate ```python # tests/eval/test_deid_leakage_gate.py import pytest from openmed.eval import run_suite, ReleaseGate, RELEASABLE RECALL_FLOOR = 0.99 # direct-identifier recall floor HELD_OUT = "eval/heldout/phi_synthetic.json" # SYNTHETIC, committed @pytest.fixture(scope="module") def gate_report(): report = run_suite( HELD_OUT, suite="golden", model_name="OpenMed/
- When to use this skill
- Quick start — a pytest gate
- Quick start — a CLI gate
- Wire it into GitHub Actions
- Workflow
- Hand-off to / from OpenMed
- Edge cases & gotchas
- Standards & references
Produce a candidate report, then gate it. Nonzero exit blocks the job. python -m openmed.eval.release_gates \
What does the gating-deid-leakage skill do?
Add a CI gate that fails the build when an OpenMed de-identification model's recall on a held-out PHI set drops below threshold or any critical identifier leaks. Use when the user wants a pytest test or CLI step that exits nonzero on de-id regression, wants to wire OpenMed's leakage-first release gates into GitHub Actions / CI, needs a recall floor plus zero-leakage assertion against a synthetic held-out set, or wants to block merges that weaken de-identification. Trigger on \"CI gate\", \"fail the build\", \"regression test\", \"de-id recall threshold\", \"block the merge\", \"exit nonzero\",
How do I install it?
Run `npx skills add maziyarpanahi/openmed --skill gating-deid-leakage --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From maziyarpanahi/openmed, a repository with 4,851 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.