vllm

npx skills add Prism-Shadow/penguin-harness --skill vllm --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# vLLM Serving vLLM serves open-weight LLMs on local GPUs with high-throughput inference behind an OpenAI-compatible API, ready for chat and agent workloads. ## Before you start If the user's message only invokes this skill (e.g. "use vllm skill") without a concrete request, ask the user what they want. Do not run any command until the goal is clear. Ask the user which model to serve; if they have no preference, recommend the small default [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B). Also ask what context length the workload needs. vLLM needs an NVIDIA or AMD GPU. Engine choice follows the user's preference: Ollama also runs on GPUs and is the simpler default — pick vLLM for high-throughput serving, and Ollama on macOS or CPU-only machines, which vLLM does not serve. Confirm the hardware first: ```bash nvidia-smi # NVIDIA: GPU model and free VRAM (AMD ROCm: rocm-smi) python3 --version # a recent Python is required ``` The model must fit the available VRAM — model size and context length drive the serve flags below. ## Suggested workflow 1. Ask the user which model to serve; with no preference, recommend [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B)

nvidia-smi          # NVIDIA: GPU model and free VRAM (AMD ROCm: rocm-smi)
python3 --version   # a recent Python is required
python3 -m venv .venv && source .venv/bin/activate
pip install vllm
vllm serve Qwen/Qwen3.5-0.8B --port 8000
vllm serve Qwen/Qwen3.5-0.8B --enable-auto-tool-choice --tool-call-parser hermes
curl http://localhost:8000/v1/models
penguin config model add --provider custom --client-type openai \
penguin config model list   # the new entry should now be listed