AI Inference & Model Serving
AI model inference and serving. Activate when: (1) Setting up LocalAI or vLLM, (2) Configuring model serving, (3) Working with GGUF/GGML models, (4) Implementing inference pipelines, or (5) Optimizing model performance.
npx skills add majiayu000/claude-skill-registry --skill inference-flexnetos-ripple-env-3 --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# AI Inference & Model Serving ## Overview This skill covers local AI inference using LocalAI, vLLM, and other model serving frameworks for running LLMs and other AI models. ## LocalAI ### Installation ```bash # Docker (recommended) docker run -p 8080:8080 \ -v $PWD/models:/models \ localai/localai:latest-cpu # With GPU support docker run --gpus all -p 8080:8080 \ -v $PWD/models:/models \ localai/localai:latest-gpu-nvidia-cuda-12 ``` ### Model Configuration ```yaml # models/llama.yaml name: llama backend: llama-cpp parameters: model: /models/llama-2-7b-chat.Q4_K_M.gguf temperature: 0.7 top_p: 0.9 top_k: 40 context_size: 4096 threads: 4 gpu_layers: 35 # Offload layers to GPU # Template for chat template: chat: | {{.System}} {{range .Messages}} {{if eq .Role "user"}}User: {{.Content}} {{else if eq .Role "assistant"}}Assistant: {{.Content}} {{end}} {{end}} Assistant: ``` ### API Usage ```bash # Chat completion (OpenAI-compatible) curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "llama", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] }' # Text completion curl http:
- Overview
- LocalAI
- Installation
- Model Configuration
- API Usage
- Python Client
- vLLM
- Server Mode
- Python API
- GGUF/GGML Models
- Model Formats
- Quantization with llama.cpp
- Model Optimization
- GPU Memory Management
Docker (recommended) docker run -p 8080:8080 \ With GPU support docker run --gpus all -p 8080:8080 \ Chat completion (OpenAI-compatible) curl http://localhost:8080/v1/chat/completions \ Text completion curl http://localhost:8080/v1/completions \ Embeddings curl http://localhost:8080/v1/embeddings \
What does the AI Inference & Model Serving skill do?
AI model inference and serving. Activate when: (1) Setting up LocalAI or vLLM, (2) Configuring model serving, (3) Working with GGUF/GGML models, (4) Implementing inference pipelines, or (5) Optimizing model performance.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill inference-flexnetos-ripple-env-3 --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.