acl-reproducibility
Use when strengthening reproducibility evidence for an ACL paper reviewed through ACL Rolling Review, covering the Responsible NLP checklist end to end, hyperparameter and compute reporting, prompt and decoding disclosure for LLM experiments, data contamination auditing, variance across runs, and checklist-to-paper consistency.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill acl-reproducibility --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# ACL Reproducibility Use this before an ARR deadline and again at camera-ready. At ACL the reproducibility instrument is the **Responsible NLP checklist**: it is mandatory, reviewers read it alongside the paper, and ARR policy makes incorrect or misleading checklist content a desk-rejection ground. Treat it as a claims audit, not paperwork. ## The checklist as a claims audit - Section A: a real Limitations discussion and a risks discussion — reviewers are told honest limitations must not be penalized, so under-disclosing is strictly worse than disclosing. - Section B: every dataset and model you *used* needs citation, version, license, and intended-use consistency (see `acl-artifact-evaluation`). - Section C: computational experiments — parameters, budget, infrastructure, hyperparameter search, and descriptive statistics with error bars. - Section D: human annotators/participants — instructions, pay, consent, ethics-board status, demographics where relevant. - Section E: AI assistants used in research, coding, or writing. Every "yes" answer should carry a section/appendix pointer; every "N/A" should survive a hostile reading of the paper. ## Reporting floor for the modern NLP pape
- The checklist as a claims audit
- Reporting floor for the modern NLP paper
- Contamination and leakage auditing
- Variance discipline
- Consistency sweep before submission
- Degrees of reproducibility to declare
- Prompt-disclosure block that satisfies reviewers
- Cheap wins ranked by effort
- Output format
What does the acl-reproducibility skill do?
Use when strengthening reproducibility evidence for an ACL paper reviewed through ACL Rolling Review, covering the Responsible NLP checklist end to end, hyperparameter and compute reporting, prompt and decoding disclosure for LLM experiments, data contamination auditing, variance across runs, and checklist-to-paper consistency.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill acl-reproducibility --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 909 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.