LQF_Machine_Learning_Expert_Guide

What it does

The skill engages in critical discussion for ML/modeling tasks, requiring a minimum of 3 refinement rounds. It challenges proposals with fact-based critique, demands evidence and baselines before accepting solutions, and targets tasks like classification, regression, clustering, feature engineering, and hyperparameter tuning. It operates using a structured critique framework, prompting for high-risk information (data size, baseline performance, data collection details, train/test split, business metric) and guiding users through a tiered critique process. It enforces iteration cycles, self-critique, and a stepwise escalation from simple baselines to more complex models, ensuring proposed approaches are justified by evidence and comparisons to simpler alternatives. It specifies allowed tools: Read, Write, Edit, Bash, Grep, Glob, and the need to cite specific ML concerns in critiques. It also provides a detailed self-critique and user critique framework, including a Baseline Thinking phase and an Iterative Refinement Cycle with Acceptance Criteria to ensure 3 critique rounds and documented baselines. The skill’s design centers on preventing premature adoption of complex models by enforcing baselines, validation, and evidence-based decision-making.

How it works

• Triggering context: Activated by prompts related to machine learning, modeling, prediction, training, classification, regression, clustering, deep learning, and related topics.
• Critical engagement: For every proposal, it requires critique and iterative refinement with minimum 3 cycles (propose → critique → refine → repeat).
• Mandatory information: Before model selection or training, it demands answers to dummy baseline performance, data size (n_samples, n_features), target distribution, data collection details, train/test split strategy, and business metric.
• Critique depth: Uses Level 1 (Diplomatic), Level 2 (Socratic), and Level 3 (Direct) critiques to push for evidence, baselines, and avoidance of data leakage or overfitting.
• Iteration protocol: Round 1 identifies 3–5 issues and asks clarifying questions; Round 2 critiques refined approach and requests evidence; Round 3 stress-tests and compares alternatives; Acceptance Criteria require complete high-risk information, established baselines, simpler alternatives considered, and 3 critique rounds.
• Self-critique: Includes a checklist for complexity, baseline, assumptions, and evidence, plus a counter-argument framework demonstrating potential alternatives and why they might be preferable.
• Guidance content: Provides explicit code-like examples for baselines, simple models, and ablation study patterns, though the core function is to guide critique rather than execute modeling.
• Licensing and scope: Declares Apache-2.0 license and the skill’s intended domain as Code Review & Quality with safety-focused, evidence-based critique processes.

When to use it

Use this skill for ML modeling tasks requiring rigorous critique before adopting a solution, including:

• Building ML models (classification, regression, clustering, forecasting)
• Evaluating model performance and debugging
• Feature engineering and data preprocessing
• Hyperparameter tuning and model optimization
• Debugging overfitting/underfitting
• Choosing between traditional ML and deep learning approaches
• Establishing baselines and conducting ablation studies
• Performing error analysis and model validation
• Statistical modeling with predictive components

What it can touch

• Tools declared: Read, Write, Edit, Bash, Grep, Glob
• It expects users to provide data-related details and can guide the user through evaluating baselines and model choices, but does not execute code itself within this summary. It prescribes asking for the dummy baseline, data size, target distribution, and data collection timeline, as well as time-based vs random splits when relevant.

Caveats

• Notable constraints: The skill enforces structured critique cycles and baseline requirements; it does not guarantee improvement or model performance, only that decisions are evidence-based and justified via critique.
• Scope note: It does not cover deployment or production infrastructure, focusing on model design, evaluation, and validation aspects.
• Licensing: Apache-2.0