Kimi-k3-in-c provides a C99-based inference engine for a 2.78 trillion-parameter model that runs on a single CPU with 8.24 GB peak memory, using a portable, no-GPU setup and a 1.56 TB checkpoint workflow.
What it is
A portable C99 implementation for running a large-scale Kimi K3 inference model on CPU only, no BLAS, no GPU, with a claimed peak RAM of 8.24 GB and a 1.56 TB checkpoint on disk. The project ships a test suite and tooling to verify engine correctness against a PyTorch reference using a 13- and 93-layer setup. The repository advertises a 2.78-trillion-parameter model and provides steps to fetch the checkpoint and build the project.
How it works
The README describes a streaming trunk approach where the dense trunk remains resident while routed experts are streamed from disk, enabling the same model to run across different memory budgets (e.g., 8 GB to 224 GB) without changing output. It introduces a packed trunk file (109 GB) and a 96-shard checkpoint layout, with a process to pack the trunk for streaming. The system supports incremental generation by carrying KV cache and recurrent state between tokens, reducing recomputation.
Getting started
Getting started involves cloning the repo, building, and running tests without a checkpoint:
git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
cd kimi-k3-in-c
make -j # seconds. Seven C files, a compiler and OpenMP
make test # under a minute
For full usage including model download and setup, the README details a sequence: clone, doctor checks, build with make -j, make test, then fetch and pack the trunk, place model on disk, and run with the provided k3 binary. The download steps require a HuggingFace token and scripts such as download-model.sh and pack-trunk.sh.
Recent releases
Latest release: v0.1.0 (2026-08-02).
Traction
Stars: 2167 Stars in last 24 hours: 479 Forks: 355 Open issues: 7
Behind the repo
No startup or company linkage is described beyond the project repository and its open-source license (Apache-2.0).
Caveats
License: Apache-2.0 Created: 2026-08-01 Last push: 2026-08-01 Platform: Linux x86-64, C99 portable. Requires 1.56 TB checkpoint for full operation and ~109 GB trunk file after packing. The workflow includes steps requiring a HuggingFace token for download and sizable disk I/O (1.56 TB checkpoint, 109 GB trunk).