issta-experiments
Use when designing or auditing ISSTA experiments, covering real subject programs and benchmarks like Defects4J, fair tool-baseline configuration, bug-finding and coverage metrics, non-parametric comparison with effect sizes, equal-budget protocols, repeated runs, and matching evidence to the claim being made.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill issta-experiments --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# ISSTA Experiments Use this before submission when the evaluation is not yet locked. ISSTA experiments earn or lose the paper on the evaluation criterion, and the reviewer pool knows the standard subjects, baselines, and statistics — so design the study to answer the exact claim, on subjects a reviewer recognizes. ## Experiment audit - Match each claim to its evidence: a bug-finding claim needs a benchmark with ground truth, a coverage claim needs a measurement protocol, a scalability claim needs a size sweep. - Use real subjects and established benchmarks where they exist — Defects4J for Java faults, real-world CVEs for security bugs, standard fuzzing corpora — so results are comparable to prior work, not to a private subject set. - Configure baselines fairly and at an **equal budget**: same time, same seeds, same subjects. A baseline throttled to lose is the fastest way to lose a reviewer's trust. - Compare with proper statistics: non-parametric tests (e.g. Mann-Whitney U) and an effect size (e.g. Vargha-Delaney Â₁₂) rather than a single run, because testing and analysis results are stochastic and rarely normal. - Report the stochastic protocol: seeds, timeout budgets, iteration
- Experiment audit
- What experiments are for at this venue
- Claim-to-evidence design table
- Vignette: evaluating a test-generation tool
- Statistical reporting floor
- Output format
What does the issta-experiments skill do?
Use when designing or auditing ISSTA experiments, covering real subject programs and benchmarks like Defects4J, fair tool-baseline configuration, bug-finding and coverage metrics, non-parametric comparison with effect sizes, equal-budget protocols, repeated runs, and matching evidence to the claim being made.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill issta-experiments --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 909 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.