intentee/

paddler

GitHub Website

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

1.6kstars

93forks

27issues

Apache-2.0license

2024since

Star historydaily snapshots by VibeCrowd

Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).

Alternatives & relatedmatched by topic overlap

pathwaycom/

llm-app

▲ 175/wk

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

59k1.5kJupyter Notebook

BerriAI/

litellm

▲ 595/wk

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

56k10kPython

janhq/

jan

▲ 103/wk

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.

44k2.9kTypeScript

khoj-ai/

khoj

▲ 106/wk

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.

36k2.3kPython

rtk-ai/

rtk

▲ 998/wk

CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

75k4.7kRust

Hmbown/

CodeWhale

▲ 245/wk

Open-source, community-driven agent harness

40k3.5kRust

Comparepaddler vs llm-app paddler vs litellm paddler vs jan

SharePost on X LinkedIn