dl-transformer-finetune
Build transformer fine-tuning plans for classification and generation
Profile →npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill dl-transformer-finetune --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Transformer Fine-Tuning Guide ## Overview Fine-tuning pretrained transformers is the dominant paradigm in modern NLP and increasingly in vision, audio, and multimodal research. The core idea is simple: take a model pretrained on massive data, then adapt it to your specific task with a comparatively small labeled dataset. But the practical details -- which layers to freeze, which optimizer and learning rate to use, how to handle catastrophic forgetting, when to use parameter-efficient methods -- determine whether fine-tuning succeeds or fails. This guide covers the full spectrum of fine-tuning approaches: full fine-tuning for maximum performance, parameter-efficient fine-tuning (PEFT) for resource-constrained settings, and the decision framework for choosing between them. The patterns are drawn from hundreds of published papers and the Hugging Face ecosystem that supports them. Whether you are fine-tuning BERT for text classification in a domain-specific corpus, adapting a large language model with LoRA for instruction following, or building a multi-task model for your research pipeline, this guide provides the recipes you need. ## Full Fine-Tuning ### Text Classification with BER
- Overview
- Full Fine-Tuning
- Text Classification with BERT
- Learning Rate Selection Guide
- Parameter-Efficient Fine-Tuning (PEFT)
- LoRA (Low-Rank Adaptation)
- QLoRA (Quantized LoRA)
- PEFT Method Comparison
- Avoiding Catastrophic Forgetting
- Fine-Tuning Checklist for Papers
- Best Practices
- References
What does the dl-transformer-finetune skill do?
Build transformer fine-tuning plans for classification and generation
How do I install it?
Run `npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill dl-transformer-finetune --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From brycewang-stanford/Auto-Empirical-Research-Skills, a repository with 3,244 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.