digital-health-clinical-asr-build

What it does

The skill guides a curation-and-synthesis process for clinical ASR benchmarks. It prompts for specialty-aware term selection, generates natural-language sentences embedding each term, passes terms through a three-stage IPA tagging pipeline (override, Merriam-Webster lookup, then Magpie G2P), and performs QA-mode synthesis before producing a NeMo-formatted manifest plus audio files for the full Cartesian product. The final outputs are a NeMo manifest and audio referenced by that manifest, designed for scoring in the next stage.

How it works

• It operates as Stage 2 in the Clinical ASR Flywheel, with the user starting from the previous setup stage and ending with a manifest.jsonl and corresponding audio for evaluation.
• It begins with one or more clarifying questions to surface candidate terms and their entity_category, then writes term_seed.csv containing lines like: term,entity_category. The allowed categories are drug, procedure, anatomy, condition, lab, role.
• For each term, it uses /data-designer to generate 3–5 context_type variants (dictation, handoff, chart_note, history) with sentences 10–30 words long embedding the term.
• It runs a three-tier IPA tagging pipeline: first consults pronunciation_overrides.csv if a match exists, then attempts Merriam-Webster lookup to derive IPA, and finally uses Magpie G2P if the prior steps fail. Each manifest row carries ipa_source indicating override, merriam-webster, or magpie_g2p.
• Before full synthesis, it performs QA-mode synthesis: one wav per term (with first voice and clean noise) and audites the clips. For magpie_g2p terms, it suggests IPA candidates based on clinical suffix patterns and validates them live against Magpie’s en-US phoneme set. Only after user audition or explicit skip, it proceeds.
• Once IPA decisions are locked, it generates the full Cartesian product synthesis (terms × voices × noise_levels × context_types) and writes a manifest.jsonl plus per-row audio into the user’s chosen $EVAL_DIR/cycle<N>/ directory. If the cycle is large (>100 rows), it warns about potential rate-limits.

When to use it

Use when you need to build a clinical ASR benchmark with curated terms, IPA annotations, and a NeMo manifest for evaluation in Stage 3. Activate after Stage 1 completion and before Stage 3 scoring.

What it can touch

The skill relies on hosted or local pronunciation pipelines and synthesis services via inlined recipes: it references pronunciation_overrides.csv, Merriam-Webster IPA lookups, Magpie G2P, and Magpie TTS through NVCF for synthesis. It writes term_seed.csv, generated sentences, pronunciation decisions, and manifest.jsonl within the user’s chosen cycle directory.

Caveats

• Requires NVIDIA_API_KEY for hosted Magpie TTS and optional DICTIONARY_API_KEY for MW lookups. Stage 1 must have been completed first. All TTS and IPA recipes are inlined; no sibling agent skill is required.
• The workflow is gated by QA auditions; advancing to full synthesis requires explicit user confirmation or an approved skip. Large runs may trigger rate-limiting on Magpie NVCF.
• Output paths are user-controlled (e.g., $EVAL_DIR/cycle<N>/). The skill does not enforce a fixed directory structure beyond that recommendation.