scientific-critical-thinking

What it does

The skill guides an agent to evaluate scientific claims and evidence quality by applying systematic critique across methodology, biases, statistics, and evidence strength. It also covers identifying logical fallacies, providing design guidance for new studies, and using frameworks like GRADE and Cochrane ROB. Visual schematics are encouraged to accompany documents, with a workflow for generating publication-quality diagrams when needed.

How it works

• Methodology Critique: Assess study design, validity (internal, external, construct, statistical conclusion), control and blinding, and measurement quality using a structured checklist.
• Bias Detection: Identify cognitive, selection, measurement, analysis biases, and confounding; use a systematic bias review approach.
• Statistical Analysis Evaluation: Critically appraise sample size/power, appropriateness of tests, multiple comparisons corrections, p-value interpretation, effect sizes with confidence intervals, handling of missing data, and model robustness.
• Evidence Quality Assessment: Rank study designs, evaluate quality within designs, apply GRADE considerations, assess convergence of evidence, and consider contextual factors.
• Logical Fallacy Identification: Detect and name causal, generalization, authority, statistical, structural, and science-specific fallacies; explain why reasoning is flawed and what evidence would be needed.
• Research Design Guidance: Provide guidance for refining research questions, selecting designs, minimizing bias, planning samples, measurement strategies, and prespecifying analyses.
• Visual schematics: When creating documents, add diagrams via the recommended schematics workflow and use the provided command to generate figures if diagrams are not present.

The skill emphasizes concrete evaluation steps, referencing relevant principles and checklists (e.g., internal/external validity, bias types, p-value interpretation, and GRADE/ROB frameworks).

When to use it

Use this skill when evaluating research methodology and experimental design, assessing statistical validity and evidence quality, identifying biases/confounding, reviewing scientific claims, conducting systematic reviews or meta-analyses, applying GRADE or Cochrane risk of bias assessments, providing critical analysis of research papers, and planning rigorous studies.

What it can touch

• Tools: Read, Write, Edit, Bash
• References and frameworks are mentioned (GRADE, Cochrane ROB, etc.), but explicit tool executions are described in general terms rather than performing external actions.

Caveats

• License: MIT
• Declared scope focuses on critical evaluation and design guidance; does not guarantee outcomes of primary studies.
• The skill relies on applying established evaluation frameworks and does not invent new methods beyond those named (GRADE, Cochrane ROB, etc.).