unsloth-inference
Unsloth models can be deployed using native optimized inference or through production serving engines like vLLM and SGLang. Native inference is accelerated 2x via forinference(), while production serving requires merging LoRA weights into the base model.
npx skills add majiayu000/claude-skill-registry --skill unsloth-inference-cuba6112-skillfactory-0929e451 --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
--- name: unsloth-inference description: Deploying fine-tuned models for production inference using native kernel optimization, vLLM, or SGLang. Triggers: inference, serving, vllm, sglang, for_inference, model merging, openai api. --- ## Overview Unsloth models can be deployed using native optimized inference or through production serving engines like vLLM and SGLang. Native inference is accelerated 2x via `for_inference()`, while production serving requires merging LoRA weights into the base model. ## When to Use - When performing local testing or simple application deployment. - When building high-throughput production endpoints using vLLM or SGLang. - When creating OpenAI-compatible APIs for drop-in replacement in existing apps. ## Decision Tree 1. Is throughput the priority? - Yes: Merge to 16-bit and use vLLM. 2. Is VRAM for serving extremely limited? - Yes: Merge to 4-bit or use GGUF via Ollama. 3. Running simple Python inference? - Yes: Call `FastLanguageModel.for_inference(model)`. ## Workflows ### Native Optimized Inference 1. Load fine-tuned model and tokenizer using `FastLanguageModel`. 2. Call `FastLanguageModel.for_inference(model)` to enable optimized kernels. 3. Use
- Overview
- When to Use
- Decision Tree
- Workflows
- Native Optimized Inference
- Merging LoRA for vLLM Serving
- Non-Obvious Insights
- Evidence
- Scripts
- Dependencies
- References
What does the unsloth-inference skill do?
Unsloth models can be deployed using native optimized inference or through production serving engines like vLLM and SGLang. Native inference is accelerated 2x via forinference(), while production serving requires merging LoRA weights into the base model.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill unsloth-inference-cuba6112-skillfactory-0929e451 --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.