emnlp-artifact-evaluation
Use when packaging the artifacts of an EMNLP paper — datasets, annotation guidelines, prompts, evaluation code, and model outputs — as anonymous review-time evidence or public post-acceptance releases, with licensing, data statements, and the inspection order NLP reviewers actually follow.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill emnlp-artifact-evaluation --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# EMNLP Artifact Evaluation Use this for evidence packaging around an EMNLP submission. EMNLP has no separate artifact-badging committee; the artifact *is* part of the scientific claim, filed under the Responsible NLP checklist and inspected at reviewer discretion. In NLP the artifact surface is unusually broad — data, labels, prompts, outputs, and scoring code are all first-class — and each has its own failure mode. ## What NLP reviewers open, in order Reviewers with thirty minutes and suspicion follow a predictable path: | Order | Artifact | What they are checking | Cheap failure | |---|---|---|---| | 1 | Data sample | Do instances look like the paper's description? | Examples contradict claimed label definitions | | 2 | Prompt files | Do prompts match the paper's claimed setup? | Prompt contains hints the paper never mentioned | | 3 | Scoring script | Is the metric computed the standard way? | Custom normalization inflates the headline metric | | 4 | Annotation guidelines | Could these instructions produce these labels? | Guidelines answer a different question than the task | | 5 | Output dumps | Are generations as good as the excerpted ones? | Body examples are the best 5 of 50
- What NLP reviewers open, in order
- Dataset packaging: the data statement standard
- Prompts and outputs as archival objects
- Review-time anonymity vs release-time findability
- Licensing without a legal department
- Restricted and unreleasable material
- Versioning after release
- Output format
What does the emnlp-artifact-evaluation skill do?
Use when packaging the artifacts of an EMNLP paper — datasets, annotation guidelines, prompts, evaluation code, and model outputs — as anonymous review-time evidence or public post-acceptance releases, with licensing, data statements, and the inspection order NLP reviewers actually follow.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill emnlp-artifact-evaluation --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 909 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.