wsdm-experiments
Use when designing or auditing the evaluation of a WSDM paper - offline ranking and recommendation metrics with bias controls, temporal-split protocols for interaction logs, baseline selection from recent WSDM/SIGIR/KDD editions, ablations that isolate the mechanism, efficiency reporting, and online-evidence framing.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill wsdm-experiments --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# WSDM Experiments Design an evaluation that survives WSDM's mixed academic-industry PC with no rebuttal to patch holes. The venue's evaluation culture is specific: reviewers assume interaction data is biased until you control for it, assume random splits leak until you say "temporal," and assume unnamed baselines were chosen to lose. Build the section so each assumption meets its answer. ## The evidence quadrants Cover the four questions every strong WSDM evaluation answers; weak papers usually max one quadrant and ignore two: | Quadrant | Question | Typical instruments | |---|---|---| | Effectiveness | Better on the task? | nDCG/MRR/MAP@k, Recall/HR@k, AUC/logloss for CTR | | Validity | Better for the claimed reason, or an artifact? | Bias controls, leakage checks, ablations | | Efficiency | Affordable at serving time? | Latency, throughput, index/memory cost, training compute | | Robustness | Where does it break? | Cold-start slices, head/tail splits, temporal drift, adversarial cases | Effectiveness without validity is the classic WSDM rejection ("gains may be position-bias artifacts"); effectiveness without efficiency loses the industry reviewer for interactive-serving claims.
- The evidence quadrants
- Protocol decisions to fix before running anything
- Baseline selection
- Ablations that isolate the mechanism
- Efficiency and scale reporting
- Online and production evidence
- Pre-submission experiment audit
- Output format
What does the wsdm-experiments skill do?
Use when designing or auditing the evaluation of a WSDM paper - offline ranking and recommendation metrics with bias controls, temporal-split protocols for interaction logs, baseline selection from recent WSDM/SIGIR/KDD editions, ablations that isolate the mechanism, efficiency reporting, and online-evidence framing.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill wsdm-experiments --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 984 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.