recsys-reproducibility
Use when strengthening the reproducibility of an ACM RecSys paper or preparing a RecSys Reproducibility Track submission — pinning dataset versions and splits, tuning baselines under an equal budget, reporting seeds and variance, avoiding sampled-metric distortion, and structuring a reproduction study with honest divergence analysis.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill recsys-reproducibility --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# RecSys Reproducibility Reproducibility is unusually load-bearing at RecSys for two reasons. First, the venue runs a dedicated **Reproducibility Track** for papers that repeat, refute, or re-scope prior results and for datasets and frameworks that enable future reproduction. Second, the field's own literature documented how often reported recommender gains fail to survive a fair re-evaluation, so reviewers of *regular* papers read reproducibility signals as a proxy for whether the gains are real. ## Why recommender results drift: the usual suspects | Drift source | Typical symptom | Pin it by | |---|---|---| | Dataset version / filtering | "MovieLens" numbers differ across papers | Exact version id, k-core filter, and checksums | | Split protocol | Session results inflated | Temporal or leave-one-last split, not random; document it | | Baseline tuning asymmetry | Neural method "beats" everything | Equal search budget per system, grid and selected config logged | | Sampled vs full-ranking metrics | Recall/nDCG off by a lot | Rank over the full item set, or state sampling and cutoff | | Seeds and nondeterminism | ±0.005 nDCG run to run | Multiple seeds; report mean ± sd, not the bes
- Why recommender results drift: the usual suspects
- Minimum reporting block for any empirical RecSys paper
- The Reproducibility Track specifically
- Vignette: a session-model reproduction
- Degrees of reproducibility
- Output format
What does the recsys-reproducibility skill do?
Use when strengthening the reproducibility of an ACM RecSys paper or preparing a RecSys Reproducibility Track submission — pinning dataset versions and splits, tuning baselines under an equal budget, reporting seeds and variance, avoiding sampled-metric distortion, and structuring a reproduction study with honest divergence analysis.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill recsys-reproducibility --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 909 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.