fse-experiments
Use when designing or auditing ESEC/FSE empirical evaluations, covering real subject systems, fair baselines, SE-standard statistics and effect sizes, qualitative and mixed-methods rigor, contamination-aware LLM ablations, provenance for mining studies, and matching evidence to the shape of each software-engineering claim.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill fse-experiments --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# FSE Experiments Use this before submission when the empirical story is not yet locked. FSE reviewers are SE empiricists; the evaluation is where a good idea is won or lost. The organizing principle is **evidence proportional to the claim** — the study must test the thing the paper actually asserts, on subjects and baselines a skeptic would accept. ## Evaluation audit - **Match evidence to the claim shape.** A claim about *developer behavior* needs behavior data; a claim about *detection* needs a labeled ground truth; a claim about *scalability* needs runtime on realistically sized inputs. Accuracy against a proxy label is not evidence for a practice claim. - **Use real subject systems,** sampled by a stated criterion, and list them in the artifact. Toy benchmarks invite the "does this hold on real code?" reject. - **Choose fair baselines,** including the strongest prior technique and a simple-but-reasonable alternative, tuned with a documented, equal budget. An untuned baseline is a scored weakness. - **Report SE-standard statistics:** effect sizes (not just p-values), confidence intervals, appropriate tests, and corrections for multiple comparisons. Say what variance and how man
- Evaluation audit
- Claim-to-evidence design table
- Contamination-aware LLM evaluation
- Provenance floor for mining studies
- Vignette: evaluating a repair technique
- Statistical reporting floor
- Output format
What does the fse-experiments skill do?
Use when designing or auditing ESEC/FSE empirical evaluations, covering real subject systems, fair baselines, SE-standard statistics and effect sizes, qualitative and mixed-methods rigor, contamination-aware LLM ablations, provenance for mining studies, and matching evidence to the shape of each software-engineering claim.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill fse-experiments --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 909 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.