pldi-experiments
Use when designing or auditing a PLDI evaluation — choosing defensible benchmark suites and baseline compiler configurations, measuring runtime, compile time, and memory with warmup and variance discipline, running ablations that isolate the claimed mechanism, and scoping claims to the platforms measured.
Profile →npx skills add brycewang-stanford/Awesome-Journal-Skills --skill pldi-experiments --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# PLDI Experiments A PLDI evaluation answers one question: *does the claimed mechanism cause the claimed effect on programs that matter?* Everything in the design flows from making that causal link auditable. The community's shared rubric is SIGPLAN's Empirical Evaluation checklist (see `pldi-reproducibility` for the measurement hygiene); this skill covers the design choices above the hygiene layer. ## Benchmark choice is an argument, not a default - **Justify the suite** relative to the claim: an allocation optimizer needs allocation-heavy programs *and* allocation-light ones (to show no regression); a parser-facing analysis needs real grammars, not microbenchmarks. - **Use community suites where they exist** and state versions; add real-world applications when the suite is known to under-represent your phenomenon. - **List exclusions with reasons.** "We exclude two SPEC programs that use `setjmp`, which our restriction rejects (§4.4)" builds trust; silent dropping destroys it. - **Include programs your technique should *not* help.** Flat results on those are evidence the instrument works. ## Baselines that survive the PC | Weak move | Defensible move | |---|---| | Compare against
- Benchmark choice is an argument, not a default
- Baselines that survive the PC
- Ablations isolate the mechanism
- The three currencies
- Anticipated-objection pass
- Negative and neutral results
- Output format
What does the pldi-experiments skill do?
Use when designing or auditing a PLDI evaluation — choosing defensible benchmark suites and baseline compiler configurations, measuring runtime, compile time, and memory with warmup and variance discipline, running ablations that isolate the claimed mechanism, and scoping claims to the platforms measured.
How do I install it?
Run `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill pldi-experiments --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Awesome-Journal-Skills, a repository with 909 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.