vLLM Semantic Router is a Go-based programmable routing layer for Mixture-of-Models LLM deployments, enabling selection and composition of model paths across heterogeneous infrastructures.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
vLLM Semantic Router is a programmable routing layer for building Mixture-of-Models systems across heterogeneous LLM infrastructure. It evaluates request signals, user preferences, and application policies to select—or compose—the right model path for each request. It aims to improve quality, cost, latency, privacy, and safety without hard-coding routing logic into applications.
How it works
The project describes routing decisions based on signals, preferences, and policies to assemble model paths across heterogeneous compute (edge, private, cloud) while keeping data within boundaries. It supports composing personalized model paths rather than routing to a single model.
Getting started
Installation is described through an external script:
curl -fsSL https://vllm-sr.ai/install.sh | bash
For platform notes, detailed setup options, and troubleshooting, see the Installation Guide linked in the README.
Recent releases
Latest releases include v0.3.0 (2026-06-05) with container image guidance:
docker pull ghcr.io/vllm-project/semantic-router/extproc:v0.3.0
docker pull ghcr.i
Earlier release: v0.2.0 (Athena, 2026-03-10); v0.1.0 (Iris, 2026-01-05).
Traction
Stars: 5113
Behind the repo
Not provided in the facts block.
Caveats
License: Apache-2.0. Language: Go. Repository created 2025-08-26; last push 2026-08-04. Open issues: 291. Forks: 798. Description and topics indicate focus areas such as kubernetes, llmrouter, mixture-of-models, prompt-engineering, and safety.