RWKV Runner is a TypeScript tool that manages RWKV models, offers a OpenAI-compatible API interface, and automates dependencies for a lightweight executable. It provides multi-platform install, model management, and a WebUI workflow.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
RWKV Runner is a RWKV management and startup tool that automates everything with a lightweight executable and provides an interface compatible with the OpenAI API. It supports multi-platform deployments and offers features for model management, front-end and back-end separation, and OpenAI-style API access.
How it works
The project integrates model management, startup automation, and a UI/backend split to allow deploying either the client or server components independently. It provides an API compatibility layer so clients designed for OpenAI can interact with RWKV models. The README mentions automatic dependency installation and multi-level VRAM configurations, with options to switch graphics acceleration in Configs.
Getting started
Simple Deploy Example
git clone https://github.com/josStorer/RWKV-Runner
# Then
cd RWKV-Runner
python ./backend-python/main.py #The backend inference service has been started, request /switch-model API to load the model, refer to the API documentation: http://127.0.0.1:8000/docs
# Or
cd RWKV-Runner/frontend
npm ci
npm run build #Compile the frontend
cd ..
python ./backend-python/webui_server.py #Start the frontend service separately
# Or
python ./backend-python/main.py --webui #Start the frontend and backend service at the same time
# Help Info
python ./backend-python/main.py -h
Features
- RWKV model management and one-click startup.
- Front-end and back-end separation, with options to deploy front-end or back-end services separately.
- Compatible with the OpenAI API, enabling ChatGPT clients to work with RWKV by configuring the API URL.
- Automatic dependency installation and predefined VRAM/config options, including a WebGPU path for broader GPU support.
- Built-in model conversion, download management, and LoRA Finetune (Windows Only).
- Embeddings, chat/completions APIs, and a configurable set of API parameters.
Getting started (additional guidance)
The repository includes a deployment workflow that can run the backend in tandem with a separate frontend service, or start both together via a web UI flag. It also highlights the ability to deploy backend-python on a server and use the program as a client by filling the server address in Settings API URL.
Recent releases
- v1.9.12 (2026-07-07): Changes include support for newer NVIDIA driver CUDA version output to prevent invalid version strings from breaking dependency installation; note about WebView2 crashes and potential fixes.
- v1.9.11 (2026-05-08): Added albatross inference backend supporting batch inference with API compatibility; works on Windows with RTX.
- v1.9.10 (2026-02-01): Fixed inference issue for RWKV GGUF model due to llama.cpp API changes.
- v1.9.9 (2026-01-27): Added search input to Models page and support for mermaid rendering.
- v1.9.8 (2026-01-25): Disabled auto throttling of @microsoft/fetch-event-source; bumped precompiled llama.cpp Vulkan; added stop_token_ids to /completions and /chat/completions and allowed stop sequences.
Traction
6450 stars, 600 forks, 177 open issues.
Behind the repo
Not provided in the README excerpt.
Caveats
- License: MIT
- Created: 2023-05-05
- Last push: 2026-07-07
- Language: TypeScript
- Open issues imply ongoing maintenance; the RELEASE notes mention specific platform considerations (Windows WebView2, CUDA driver interactions).