LMCache is a Python KV cache layer to accelerate LLM inference, with CUDA/ROCm wheels and multiple backends. It provides installation, release options, and a description of features.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
LMCache is a KV cache management layer for LLM inference. It is vendor-neutral and can offload KV caches to a tiered storage hierarchy, with observability metrics and pluggable backends.
How it works
LMCache runs as a standalone daemon to manage KV caches independently from the inference engine. It supports persistent offloading across CPU RAM, local storage, and remote backends, and offers a SERDE interface for KV transformation. It includes non-prefix KV reuse via CacheBlend and transport layers for KV transfer.
Getting started
To install LMCache, use:
pip install lmcache
For more setup options, see:
- Installation
- Quickstart
- LMCache Recipes
- CLI Reference
- Benchmarking Guide
- Production Deployment
## Recent releases
Latest releases include v0.5.3rc3 with CUDA 12.9 wheels and ROCm wheels for AMD Instinct, and nightly CUDA 12.9/13.0 wheels:
- v0.5.3rc3 CUDA 12.9 wheel: CUDA 12.9 (2026-08-05)
- v0.5.3rc3 ROCm wheel: gfx942, gfx950 (AMD) (2026-08-05)
- nightly CUDA 12.9: Nightly 2026-08-04 (CUDA 12.9) (2026-08-04)
- nightly CUDA 13.0: Nightly 2026-08-04 (CUDA 13.0) (2026-08-04)
Traction
Stars: 11017
## Behind the repo
Linked startup/company: Tensormesh mentioned in Adoption/Partnerships.
## Caveats
License: Apache-2.0
Created: 2024-05-28
Last push: 2026-08-04
Open issues: 472