SIE is an open-source self-hosted inference engine and production cluster that serves 100+ models via a single API, with integrations and a production-like stack. It supports embeddings, generation, and retrieval tasks across multiple models and frameworks.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
SIE: Superlinked Inference Engine. Open-source inference server and production cluster for all the models your agent needs. OpenAI-compatible API endpoints are exposed, and it serves multiple models simultaneously with on-demand loading and LRU eviction. It includes a pre-configured model catalog and integrates with common frameworks and vector stores.
How it works
The system runs a cluster that can host many models on demand, loading weights as needed and evicting least-recently-used models. It exposes API endpoints for embeddings, chat completions, completions, and responses via an OpenAI-compatible API. It ships a full production stack including a load-balancing gateway, KEDA autoscaling, Grafana dashboards, and Terraform for cloud deployments. Integrations span LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB.
Getting started
From the README:
"The server speaks the OpenAI API out of the box, embeddings and generation alike (the cluster gateway serves /v1/chat/completions, /v1/completions, and /v1/responses)."
Install and verify workspace members:
uv python install 3.12
uv lock --check
uv sync --frozen --all-packages
uv run --frozen --project . --no-sync pytest -c pyproject.toml
Start the server examples:
pip install "sie-server[local]" && sie-server serve
Or use CPU Docker image:
docker run -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cpu-default
Test readiness and a sample embedding call:
curl http://localhost:8080/readyz # expect: ok
curl http://localhost:8080/v1/embeddings \
-H 'Content-Type: application/json' \
-d '{"model": "sentence-transformers/all-MiniLM-L6-v2", "input": "Hello world"}'
# {"object": "list", "data": [{"object": "embedding", "embedding": [-0.0344, 0.0310, ...}]}'
Getting started (continued)
SDKs:
pip install sie-sdk # Python
npm install @superlinked/sie-sdk # TypeScript (pnpm and yarn work too)
Example client usage demonstrates encoding, scoring, and entity extraction using provided models and labels.
Recent releases
Latest releases include v0.6.26 (2026-08-02) and v0.6.25 (2026-07-30), with notes about cache, embedding caps, and model artifacts. Other entries show incremental fixes and feature work such as immutable Docling artifacts and gateway/documentation updates.
Traction
Stars: 2396
Behind the repo
N/A
Caveats
License: Apache-2.0. Open issues: 9. Created: 2023-11-07. Last push: 2026-08-04. Language: Python. Tags include bge, colbert, data-pipeline, deep-learning, embeddings, inference, inference-server, information-retrieval, llm, ml, mlops, natural-language-processing, nlp, python, reranking, retrieval, retrieval-augmented-generation, semantic-search, splade, vector-search.