llama-cpp
Run GGUF models directly, load LoRA adapters, benchmark inference speed, and serve models via llama-server using llama.cpp. Includes Qwen 3.5 serve scripts (9B dense + F16, 35B MoE) with asymmetric KV cache and thinking mode. Secondary to Ollama; use when needing direct model control or LoRA hot-loading. Triggers on 'llama.cpp', 'GGUF', 'LoRA adapter', 'benchmark inference', 'llama-server'.
npx skills add majiayu000/claude-skill-registry --skill llama-cpp-tdimino-claude-code-minoan --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# llama.cpp - Secondary Inference Engine Direct access to llama.cpp for faster inference, LoRA adapter loading, and benchmarking on Apple Silicon. Ollama remains primary for RLAMA and general use; llama.cpp is the power tool. ## Prerequisites ```bash brew install llama.cpp ``` Binaries: `llama-cli`, `llama-server`, `llama-embedding`, `llama-quantize` ## Quick Reference ### Resolve Ollama Model to GGUF Path To avoid duplicating model files, resolve an Ollama model name to its GGUF blob path: ```bash ~/.claude/skills/llama-cpp/scripts/ollama_model_path.sh qwen2.5:7b ``` ### Run Inference ```bash GGUF=$(~/.claude/skills/llama-cpp/scripts/ollama_model_path.sh qwen2.5:7b) llama-cli -m "$GGUF" -p "Your prompt here" -n 128 --n-gpu-layers all --single-turn --simple-io --no-display-prompt ``` ### Start API Server To start an OpenAI-compatible server (port 8081, avoids Ollama's 11434): ```bash ~/.claude/skills/llama-cpp/scripts/llama_serve.sh <model.gguf> # Or with options: PORT=8082 CTX=8192 ~/.claude/skills/llama-cpp/scripts/llama_serve.sh <model.gguf> ``` Test the server: ```bash curl http://localhost:8081/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model":"default",
- Prerequisites
- Quick Reference
- Resolve Ollama Model to GGUF Path
- Run Inference
- Start API Server
- Serve Qwen3.5
- Benchmark (llama.cpp vs Ollama)
- LoRA Adapter Inference
- Convert Kothar LoRA to GGUF
- When to Use llama.cpp vs Ollama
- Architecture
- Subprocess Best Practices (Build 8180+)
brew install llama.cpp llama-cli -m "$GGUF" -p "Your prompt here" -n 128 --n-gpu-layers all --single-turn --simple-io --no-display-prompt Or with options: curl http://localhost:8081/v1/chat/completions \ Full precision F16 (~17.9 GB, zero quantization loss) Non-thinking mode, shorter context Download once (~17.9 GB) huggingface-cli download unsloth/Qwen3.5-9B-GGUF "Qwen3.5-9B-BF16.gguf" --local-dir ~/models Serve F16 python3 ~/.claude/skills/llama-cpp/scripts/convert_lora_to_gguf.py \
What does the llama-cpp skill do?
Run GGUF models directly, load LoRA adapters, benchmark inference speed, and serve models via llama-server using llama.cpp. Includes Qwen 3.5 serve scripts (9B dense + F16, 35B MoE) with asymmetric KV cache and thinking mode. Secondary to Ollama; use when needing direct model control or LoRA hot-loading. Triggers on 'llama.cpp', 'GGUF', 'LoRA adapter', 'benchmark inference', 'llama-server'.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill llama-cpp-tdimino-claude-code-minoan --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.