ai-llm-inference
Operational patterns for LLM inference: latency budgeting, tail-latency control, caching, batching/scheduling, quantization/compression, parallelism, and reliable serving at scale. Emphasizes production-grade performance, cost control, and observability.
npx skills add majiayu000/claude-skill-registry --skill ai-llm-inference-vasilyu1983-ai-agents-public --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# LLMOps – Inference & Optimization – Production Skill Hub **Modern Best Practices (January 2026)**: - Treat inference as a **systems problem**: SLOs, tail latency, retries, overload, and cache strategy. - Use **continuous batching / smart scheduling** when serving many concurrent requests (Orca scheduling: https://www.usenix.org/conference/osdi22/presentation/yu). - Use **KV-cache aware serving** (PagedAttention/vLLM: https://arxiv.org/abs/2309.06180) and **efficient attention kernels** (FlashAttention: https://arxiv.org/abs/2205.14135). - Use **speculative decoding** when latency is critical and draft-model quality is acceptable (speculative decoding: https://arxiv.org/abs/2302.01318). - Quantize only with **measured** quality impact and rollback plan (quantization must be validated on your eval set). This skill provides **production-ready operational patterns** for optimizing LLM inference performance, cost, and reliability. It centralizes **decision rules**, **optimization strategies**, **configuration templates**, and **operational checklists** for inference workloads. No theory. No narrative. Only what Claude can execute. --- ## When to Use This Skill Claude should activate t
- When to Use This Skill
- Scope Boundaries (Use These Skills for Depth)
- Quick Reference
- Decision Tree: Inference Optimization Strategy
- Core Concepts & Practices
- Core Concepts (Vendor-Agnostic)
- Implementation Practices (Tooling Examples)
- Do / Avoid
- Resources (Detailed Operational Guides)
- Infrastructure & Serving
- Performance Optimization
- Deployment & Operations
- Templates
- Inference Configs
What does the ai-llm-inference skill do?
Operational patterns for LLM inference: latency budgeting, tail-latency control, caching, batching/scheduling, quantization/compression, parallelism, and reliable serving at scale. Emphasizes production-grade performance, cost control, and observability.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill ai-llm-inference-vasilyu1983-ai-agents-public --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.