Chitu is a Python-based high-performance large language model inference framework focused on efficiency, flexibility, and availability. It supports multi-node execution and various model deployments, with releases adding features like chitu.run and new models.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Chitu「赤兔」is a high-performance large model inference framework focused on efficiency, flexibility, and availability. It targets production-grade deployment across CPU, single-GPU, and large clusters, with multi-model and multi-hardware support.
How it works
The project provides an inference engine that integrates and optimizes components for diverse hardware and models. Release notes indicate support for multi-node, multi-instance deployments, PD separation, and various optimized operators and toolchains across models and platforms.
Getting started
Install and usage guidance suggest using chitu.run for deployment and refer to the Releases assets page. See:
- "建议使用 chitu.run 进行部署，请从 Releases 中的 Assets 页面下载。"
Also, consult the Development Manual for complete installation and usage instructions, as cited:
- "请参阅开发手册获取完整的安装使用说明。"
View supported models for specifics:
- "更多模型请参见 支持的模型。"
Recent releases
Latest: v0.6.0 (2026-07-02) introduces chitu.run single executable to start multi-node/multi-instance/PD-separated tasks, GLM-5.2 model, Tong platforms support, and new features:
- 新增 chitu.run：单可执行文件即可启动多节点、多实例、PD 分离等复杂任务。
- 新模型：GLM-5.2。
- 昇腾 910B 平台支持 GLM-5 系列。
- 支持在请求中设置
reasoning_effort。 - 增加 DeepSeek-V4 工具调用支持。
- INT8 W8A8 MoE 增加 triton 算子实现。
Previous releases include v0.5.7, v0.5.6, v0.5.5, and v0.5.4 with model additions, performance optimizations, and integrations. See the list in the Releases section for details.
Traction
Stars: 3153 Forks: 273 Open issues: 60
Behind the repo
No startup/company link is provided in the facts.
Caveats
License: Apache-2.0 Created: 2025-02-20 Last push: 2026-08-04 Language: Python Topics include: deepseek, gpu, llm, llm-serving, model-serving, pytorch
Licensing and third-party notices are mentioned in the repository. No explicit user-facing caveats beyond standard open-source usage are provided in the provided facts.