Inference & Serving

generative-computing/

granite-switch

Granite Switch — Build AI models like you build software

PMZFX/

intel-arc-pro-b70-benchmarks

Benchmark results and performance data for the Intel Arc Pro B70 GPU (Xe2/Battlemage) - LLM inference, video generation, dual-GPU scaling.

MADEVAL/

FerryAI

Native AI inference for PHP 8.3+ - run ONNX, GGUF (llama.cpp) and RubixML models directly in your PHP process via FFI. Chat, streaming, embeddings, RAG and vector search. No Python, no HTTP microservices, no Docker sidecars.

OpenCPIL/

prima.cpp

[Official] prima.cpp: Fast 30-70B LLM inference on heterogeneous and everyday home devices

wuxuedaifu/

vllm-chatterbox-stream

OpenAI-compatible multilingual TTS server — Chatterbox on vLLM with real-time PCM audio streaming, low time-to-first-byte (~0.7 s), voice cloning, and 23 languages.

AdamBien/

lightmetal

Apple Silicon mlx with Zero Dependency Java