LQF_Machine_Learning_Expert_Guide

What it does

The skill engages in critical discussion mode for ML projects, requiring at least three refinement rounds. It prompts the agent to demand a dummy baseline, data size, target distribution, and data collection details before proceeding. It enforces explicit baselines, simple vs. complex model comparisons, and ablation checks. It uses a tiered critique approach (diplomatic, Socratic, direct) and mandates evidence-based reasoning with baseline lifts and potential data leakage checks. It prescribes stepwise progression from problem framing to data engineering and modeling, prioritizing simpler, interpretable baselines and only increasing complexity when justified by measurable improvement. It imposes mandatory information checks for high-risk decisions and includes a self-critique framework to assess complexity, baselines, assumptions, and evidence. It also provides structured critique templates and examples to challenge proposals and ensure robust evaluation. The skill signals that it operates within a critical engagement protocol and requires minimum three iteration cycles, evidence-based critique, and validation against baselines before acceptance. Commands, file names, and tool names are quoted exactly as described, and all steps are described concretely only if explicitly stated in the skill.

How it works

The skill defines a multi-round critique protocol: Round 1 proposes an approach and identifies 3–5 potential issues, asks clarifying questions, and suggests simpler alternatives. Round 2 critiques the refined approach, requests evidence (baseline, data distribution), and proposes counter-examples. Round 3 stresses edge cases, compares against alternatives, and delivers a final critique before acceptance. During critiques, it enforces mandatory information checks for high-risk decisions, including data size (n_samples, n_features), dummy baseline performance, target distribution, data collection details, train/test split strategy, and business metrics. It also provides a self-critique checklist covering complexity, baseline justification, assumptions, and evidence, plus a devil’s advocate section that immediately provides counter-arguments to each suggestion. The pattern includes a Simplicity Ladder of alternatives (No ML, Dummy Baseline, Simple Linear, Simple Non-Linear, Complex) and a Phase-based workflow: Problem Definition, Baseline Thinking, Data Engineering, Modeling, and Ablation. It concludes with a summary to ensure the approach has baseline justification and 3 refinement rounds.

When to use it

Use this skill to drive critical evaluation of ML/modeling proposals, especially when high-stakes decisions or complex modeling are involved. It should be applied when there is a need to establish baselines, test for data leakage, verify the necessity of complexity, and require evidence of lift over baselines before adopting advanced models. It is geared toward tasks involving ML classification, regression, clustering, feature engineering, hyperparameter tuning, and model validation, with a focus on transparent justification and rigorous critique.

What it can touch

Not applicable beyond the described workflow. The skill references using baselines, comparing models, and requesting information; no external touchpoints or explicit tool invocations are listed in the provided content beyond the general critique framework.

Caveats

The description emphasizes iterative critique and evidence-based decisions but does not guarantee model performance, baselines, or data quality. It requires explicit input from the user (data characteristics, baselines, split strategy) and enforces critique cycles before any acceptance. License, dependencies, and implementation specifics are not stated beyond the critique protocol.