Trending AI repositories

vllm-project/

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

sgl-project/

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

openvinotoolkit/

openvino

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

superduper-io/

superduper

Superduper: End-to-end framework for building custom AI applications and agents.

gpustack/

gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

microsoft/

aici

AICI: Prompts as (Wasm) Programs

xLLM-AI/

xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

StarlightSearch/

EmbedAnything

Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀

jmaczan/

tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

openinfer-project/

openinfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

agentic-in/

inferoa

Inference-native Tokenmaxxing Agent Harness for Loop Engineering

MADEVAL/

FerryAI

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embeddings, RAG and vector search. No Python, no HTTP microservices, no Docker sidecars.