moai-ml-llm-fine-tuning
Enterprise LLM Fine-Tuning with LoRA, QLoRA, and PEFT techniques
npx skills add majiayu000/claude-skill-registry --skill moai-ml-llm-fine-tuning-jg-chalk-io-nora-livekit --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# LLM Fine-Tuning Expert **Parameter-Efficient Fine-Tuning (PEFT) for Enterprise LLMs** > **Focus**: LoRA, QLoRA, Domain Adaptation > **Models**: Llama 3.1, Mistral, Mixtral, Falcon > **Stack**: PyTorch, Transformers, PEFT, bitsandbytes --- ## Overview Enterprise-grade fine-tuning strategies for customizing Large Language Models (LLMs) with minimal resource requirements. ### Core Capabilities - **Parameter-Efficient Fine-Tuning (PEFT)**: LoRA, QLoRA, Prefix Tuning - **Quantization**: 4-bit/8-bit training with bitsandbytes - **Distributed Training**: Multi-GPU, DeepSpeed, FSDP - **Optimization**: Flash Attention 2, Gradient Checkpointing - **Evaluation**: Perplexity, BLEU, ROUGE, Domain benchmarks ### Technology Stack - **PEFT 0.13+**: Adapter management - **Transformers 4.45+**: Model architecture - **TRL 0.11+**: Supervised Fine-Tuning (SFT), DPO - **Accelerate 0.34+**: Training loop orchestration - **bitsandbytes 0.45+**: Low-precision optimization --- ## Fine-Tuning Strategies | Method | Params Updated | VRAM (70B) | Use Case | | -------------------- | -------------- | ---------- | -------------------------------------- | | **Full Fine-Tuning** | 100% | ~420GB | Foundation model
- Overview
- Core Capabilities
- Technology Stack
- Fine-Tuning Strategies
- Implementation Patterns
- 1. QLoRA Configuration (Recommended)
- 2. Training Loop with TRL
- 3. Data Preparation
- Advanced Techniques
- Multi-GPU Distributed Training
- Model Merging
- Validation Checklist
- Related Skills
config.yaml for accelerate accelerate launch --config_file config.yaml train.py
What does the moai-ml-llm-fine-tuning skill do?
Enterprise LLM Fine-Tuning with LoRA, QLoRA, and PEFT techniques
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill moai-ml-llm-fine-tuning-jg-chalk-io-nora-livekit --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.