llm-gold-bound-failure-check
Diagnose whether an LLM classifier's validation-gate failure is GOLD-BOUND before spending on prompt revision or model changes. Use when: (1) a scoring pipeline over-predicts a label (precision low, recall high) and a prompt clarification is proposed to tighten it, (2) a pilot/validation gate fails and the fix candidates are prompt edits, (3) inter-rater agreement on the weak label was already low (κ < ~0.6). Core check: if gold POSITIVES share the exact feature the revision would exclude, no prompt can pass a gold-scored gate — recall craters while precision barely moves. Also documents the v
npx skills add kennethkhoocy/applied-micro-skills --skill llm-gold-bound-failure-check --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# LLM Gold-Bound Failure Check ## Problem When an LLM scoring pipeline over-predicts one label, the reflex fix is a prompt clarification ("score positive ONLY when..."). But if the gold standard itself does not separate the texts you want excluded from the texts it labels positive, the revision removes true and false positives together. The pilot fails, the spend is wasted, and — worse — an un-gated adoption would have silently destroyed recall in production. ## Context / Trigger Conditions - A domain/label shows precision ≪ recall (e.g. P 0.46 / R 0.96) against gold - A prompt edit is proposed to exclude a specific text type (boilerplate, affirmative-program language, non-risk framing) - The label's gold council/inter-rater agreement was already the weakest (κ below ~0.6 is the warning sign that the construct is contested) ## Solution **Step 0 — the ~$0 check, BEFORE building anything:** read a sample of gold POSITIVES for the weak label and ask: do they contain the feature the revision would exclude? Compare them side-by-side with the false positives. - Gold positives and false positives are the same kind of text → the failure is **gold-bound**. Stop. No prompt passes a gold-scor
- Problem
- Context / Trigger Conditions
- Solution
- Verification
- Example
- Notes
What does the llm-gold-bound-failure-check skill do?
Diagnose whether an LLM classifier's validation-gate failure is GOLD-BOUND before spending on prompt revision or model changes. Use when: (1) a scoring pipeline over-predicts a label (precision low, recall high) and a prompt clarification is proposed to tighten it, (2) a pilot/validation gate fails and the fix candidates are prompt edits, (3) inter-rater agreement on the weak label was already low (κ < ~0.6). Core check: if gold POSITIVES share the exact feature the revision would exclude, no prompt can pass a gold-scored gate — recall craters while precision barely moves. Also documents the v
How do I install it?
Run `npx skills add kennethkhoocy/applied-micro-skills --skill llm-gold-bound-failure-check --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From kennethkhoocy/applied-micro-skills, a repository with 54 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.