scientific-critical-thinking

What it does

The skill guides a user to critically assess scientific claims and evidence quality across methodology, bias, statistics, and evidence grading, applying frameworks like GRADE and Cochrane ROB, and provides structured checks for evaluating study design, validity, measurement quality, and reasoning fallacies.

How it works

• It outlines six core capabilities: Methodology Critique, Bias Detection, Statistical Analysis Evaluation, Evidence Quality Assessment, Logical Fallacy Identification, and Research Design Guidance.
• For Methodology Critique, it provides a step-by-step evaluation: Study Design Assessment, Validity Analysis (internal, external, construct, statistical conclusion), Control and Blinding, and Measurement Quality.
• For Bias Detection, it structures systematic review across cognitive, selection, measurement, analysis, and confounding biases, with prompts to examine study processes and reporting.
• For Statistical Analysis Evaluation, it supplies a checklist on sample size, test appropriateness, multiple comparisons, p-values, effect sizes and CIs, missing data, regression modeling, and common pitfalls.
• For Evidence Quality Assessment, it ranks study designs, assesses quality within design type, applies GRADE considerations, evaluates convergence of evidence, and accounts for contextual factors.
• For Logical Fallacy Identification, it lists fallacy categories (causation, generalization, authority, statistical, structural, science-specific) and how to name and check them.
• For Research Design Guidance, it details designing studies: refining questions, design selection, bias minimization, sample planning, measurement strategy, and analysis planning, with prespecification and rigor emphasis.

When to use it

• Use when evaluating research methodology and experimental design
• Use when assessing statistical validity and evidence quality
• Use when identifying biases and confounding in studies
• Use when reviewing scientific claims and conclusions
• Use when conducting systematic reviews or meta-analyses
• Use when applying GRADE or Cochrane risk of bias assessments
• Use when providing critical analysis of research papers

What it can touch

• Tools listed: Read, Write, Edit, Bash
• Frameworks referenced: GRADE, Cochrane ROB
• It discusses generating schematics via a separate workflow (scientific schematics), including a Python command line example for generating diagrams

Caveats

• The skill references external reference documents and frameworks (e.g., references/scientific_method.md, references/common_biases.md, references/statistical_pitfalls.md, references/evidence_hierarchy.md) but does not reproduce their content here
• It emphasizes explicit steps and structured prompts without guaranteeing specific study outcomes or results
• Licensed under MIT license