acl-experiments
Use when designing or auditing experiments for an ACL paper, covering tuned LLM baselines, multi-dataset and multilingual evaluation, statistical significance and variance, human evaluation with agreement reporting, contamination and prompt-sensitivity controls, ablations, and error-analysis expectations in NLP reviewing.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill acl-experiments --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# ACL Experiments Use this while the experimental story can still change. The ACL evidence bar is not "beats the baseline once": it is a defensible measurement of a language capability, with the failure modes examined. ## Baseline honesty - Include the strongest cheap baseline: a well-prompted current LLM has become mandatory context for most tasks — a method beating only pre-LLM systems invites the "does this matter now?" review. - Tune baselines with the same care as your method (same search budget, same data); reviewers explicitly probe for asymmetric tuning. - Report the trivial baselines (majority class, copy input, retrieval-only) when they contextualize how hard the task actually is. ## Evaluation design - Breadth must match the claim: a "general" claim needs multiple datasets; a cross-lingual claim needs typologically distinct languages, not three Romance neighbors. - Automatic metrics need justification for generation tasks — pair n-gram or embedding metrics with human or LLM-judge evaluation, and validate any LLM-judge against human labels before leaning on it. - Fix the evaluation protocol before final runs: dev-set peeking on the test set via repeated submissions is unr
- Baseline honesty
- Evaluation design
- Statistical floor
- Contamination and validity controls
- Ablations and the mechanism claim
- Error analysis as a deliverable
- Pre-run design worksheet
- Common evidence failures seen in ARR reviews
- When compute is the constraint
- Output format
What does the acl-experiments skill do?
Use when designing or auditing experiments for an ACL paper, covering tuned LLM baselines, multi-dataset and multilingual evaluation, statistical significance and variance, human evaluation with agreement reporting, contamination and prompt-sensitivity controls, ablations, and error-analysis expectations in NLP reviewing.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill acl-experiments --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 909 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.