Mooncake provides a disaggregated KVCache-based architecture for large-scale LLM serving, featuring a Transfer Engine and distributed KV cache store with multi-tier caching and cross-node data movement. It integrates with SGLang and vLLM and has active releases and extensive ecosystem adoption.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. It is an infrastructure project for large-scale LLM inference and training, featuring a KV cache-centric disaggregated architecture that separates prefill and decode clusters, leveraging CPU, DRAM, and SSD resources in GPU clusters to build a disaggregated KV cache pool. It includes a high-performance Transfer Engine for low-latency data movement, Mooncake Store for distributed KV cache and model-weight management, and Mooncake EP & PG for elastic MoE serving. It is deeply integrated with ecosystems such as SGLang and vLLM.
How it works
Mooncake comprises several components:
- Transfer Engine (TE): a high-performance data transfer framework that supports batched movement across storage, network, and accelerators, with multiple transport protocols, topology-aware routing, multi-NIC bandwidth aggregation, and automatic failover. TE enables low-latency, scalable data transmission for distributed AI workloads. Highlights include multi-RDMA NIC usage for bandwidth aggregation, topology-aware path selection, automatic failover, and performance at scale.
- Mooncake Store: a distributed KV cache storage engine built on TE, managing reusable KV caches and model weights with support for high-bandwidth data transfer, object storage, replication, and eviction, plus multi-tier caching across DRAM and SSD/NVMe.
Highlights from the Transfer Engine section describe efficiency with multiple RDMA NIC devices, topology-aware path selection, robustness against temporary network errors, and high throughput (up to 87 GB/s and 190 GB/s on specified RoCE configurations; 2.4x and 4.6x faster than TCP).
Getting started
The repository provides setup and integration guidance via the project documentation and design guides. The README links to the Transfer Engine guide and Mooncake Store guide for detailed configuration and usage. The latest releases indicate ongoing maintenance and feature additions (e.g., RDMA QP path diversity, runtime environment changes, and store-related updates).
Commands to reference exactly as written in the README are not fully included in the truncated text provided, but the README and release notes indicate a workflow around building and configuring TE and Mooncake Store, with references to related documentation:
- Transfer Engine guide: https://kvcache-ai.github.io/Mooncake/design/transfer-engine/index.html
- Mooncake Store guide: https://kvcache-ai.github.io/Mooncake/design/mooncake-store.html
Recent releases
- v0.3.12.post1 v0.3.12.post1 (2026-07-25):
- [build] Strip shared libraries to reduce NPU wheel size by @JieTang66 in https://github.com/kvcache-ai/Mooncake/pull/2202
- [Docs] Update README with citation details by @UNIDY2002
- v0.3.12 v0.3.12 (2026-07-23):
- docs: reflect WITH_NVIDIA_PEERMEM change from CMake flag to runtime env var by @stmatengss with @Copilot in https://github.com/kvcache-ai/Mooncake/pull/2164
- feat(rdma): mlx5dv QP
- v0.3.11.post1 v0.3.11.post1 (2026-05-23):
- docs: reflect WITH_NVIDIA_PEERMEM change from CMake flag to runtime env var by @Copilot in https://github.com/kvcache-ai/Mooncake/pull/2164
- feat(rdma): mlx5dv QP path diversity v
- v0.3.11 v0.3.11 (2026-05-21):
- Fix/tent batch transfer merge boundary by @Primary33 in https://github.com/kvcache-ai/Mooncake/pull/1704
- docs: add TorchSpec in Mooncake README by @zhyncs in https://github.com/k
- v0.3.10.post2 v0.3.10.post2 (2026-04-22):
- [Store] support resolving master RPC address from interface by @YiXR in https://github.com/kvcache-ai/Mooncake/pull/1784
- Bump google.golang.org/grpc from 1.59.0 to 1.79.3 in /moo
Traction
Stars: 6151
(Only raw numbers are provided as per the data: stars_7d or stars_1d are not present in the FACTS.)
Behind the repo
Mooncake is associated with Moonshot AI and Kimi, as described in the README; it is an open-source project with multiple ecosystem integrations (SGLang, vLLM, TensorRT-LLM, NIXL, etc.).
Caveats
License: Apache-2.0 Created: 2024-06-25 Last push: 2026-08-04 Open issues: 472 Languages: C++ Repository age and activity noted via last_push date and releases. The README references are to various guides and integration stories, but no explicit usage commands are included in the truncated text here. For precise install and usage commands, refer to:
- https://kvcache-ai.github.io/Mooncake/design/transfer-engine/index.html
- https://kvcache-ai.github.io/Mooncake/design/mooncake-store.html
- The latest release notes listed above