Forge is a Python framework that enables self-hosted LLM tool-calling with guardrails, supporting multiple backends and a proxy mode for integration with existing harnesses.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows. It provides guardrails, rescue parsing, retry logic, and tooling integration to manage tool calls within an agent loop. It supports multiple backends (llama-server, Ollama, Llamafile, vLLM, Anthropic) and offers a drop-in proxy server as well as a WorkflowRunner and Guardrails middleware.
How it works
Forge exposes three usage modes:
- Proxy server: sits between a client and a local model server, validating tool calls, rescuing misformatted tool calls, retrying on errors, and injecting a synthetic respond tool when needed. It can proxy OpenAI chat-completions and Anthropic Messages endpoints.
- WorkflowRunner: defines tools and backend, runs structured agent loops, handles system prompts, context, and guardrails across multi-turn workflows.
- Guardrails middleware: modular guardrails inside your own orchestration loop to validate and manage tool calls.
Backends supported: Ollama, llama-server, Llamafile, vLLM, and Anthropic.
Getting started
Install:
pip install forge-guardrails # core only
pip install "forge-guardrails[anthropic]" # + Anthropic client
Backend setup examples (llama-server, Ollama, Anthropic) are provided in the README, and a Quick Start shows how to run a Llamafile client with a local gguf model. The README includes a Quick Start Python snippet demonstrating a simple Workflow, ToolDef, and LlamafileClient usage, plus guidance to run the proxy and configure clients.
Getting started commands (examples from README):
llama-server -m path/to/Ministral-3-8B-Instruct-2512-Q8_0.gguf --jinja -ngl 999 --port 8080
import asyncio
from pydantic import BaseModel, Field
from forge import (
Workflow, ToolDef, ToolSpec,
WorkflowRunner, LlamafileClient,
ContextManager, TieredCompact,
)
Recent releases
- v0.8.3 (2026-08-03): proxy model routing and deterministic eval publication. Forge restores model identity handling across vLLM and Anthropic proxy paths while making evaluation collection and pub.
- v0.8.2 (2026-07-28): expanded model evals and dashboard refresh. Hardened llama.cpp 500 recovery; added compatibility with llama-server unversioned chat endpoint.
- v0.8.1 (2026-07-10): rescue tool calls from malformed llama.cpp 500s; bug-fix for llamafile backend.
- v0.8.0 (2026-06-27): first-class authentication across all proxy modes and backends; forwards a single credential to the backend via --backend-api-key or --gguf/--model-path flow.
- v0.7.6 (2026-06-20): Ollama backend fixes and inline reasoning capture; multi-turn tool sessions improved.
Traction
Stars: 2211 (as of the provided data).
Behind the repo
No linked startup/company information provided in the FACTS.
Caveats
License: MIT. Created: 2026-02-16. Last push: 2026-08-03. Open issues: 4. Language: Python.