text-generation-inference
Deploy LLMs with Hugging Face Text Generation Inference. Configure quantization, continuous batching, and tensor parallelism. Use for production LLM serving, high-throughput inference, and model deployment.
npx skills add majiayu000/claude-skill-registry --skill text-generation-inference --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Text Generation Inference (TGI) Expert guidance for Hugging Face's production LLM inference server. ## Installation ### Docker ```bash # Basic GPU deployment docker run --gpus all -p 8080:80 \ -v /path/to/models:/data \ ghcr.io/huggingface/text-generation-inference:latest \ --model-id meta-llama/Llama-3.1-8B-Instruct # With quantization docker run --gpus all -p 8080:80 \ -v /path/to/models:/data \ ghcr.io/huggingface/text-generation-inference:latest \ --model-id meta-llama/Llama-3.1-8B-Instruct \ --quantize bitsandbytes-nf4 # Multi-GPU docker run --gpus all -p 8080:80 \ -v /path/to/models:/data \ ghcr.io/huggingface/text-generation-inference:latest \ --model-id meta-llama/Llama-3.1-70B-Instruct \ --num-shard 4 ``` ### Docker Compose ```yaml services: tgi: image: ghcr.io/huggingface/text-generation-inference:latest ports: - "8080:80" volumes: - ./models:/data environment: - HUGGING_FACE_HUB_TOKEN=${HF_TOKEN} command: > --model-id meta-llama/Llama-3.1-8B-Instruct --max-input-length 4096 --max-total-tokens 8192 --max-batch-prefill-tokens 4096 deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: [gpu] ``` ## Server Options ```bash text-generation-launch
- Installation
- Docker
- Docker Compose
- Server Options
- Quantization Options
- API Usage
- Python Client
- OpenAI-Compatible
- REST API
- Embedding Support
- Speculative Decoding
- Performance Tuning
- Kubernetes Deployment
- Monitoring
Basic GPU deployment docker run --gpus all -p 8080:80 \ With quantization Multi-GPU text-generation-launcher \ BitsAndBytes 4-bit GPTQ AWQ EETQ (efficient 8-bit) FP8
What does the text-generation-inference skill do?
Deploy LLMs with Hugging Face Text Generation Inference. Configure quantization, continuous batching, and tensor parallelism. Use for production LLM serving, high-throughput inference, and model deployment.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill text-generation-inference --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.