guardrails-llm-filter

What it is

guardrails-llm-filter is a transparent reverse proxy that masks sensitive data in requests to LLM providers and restores the originals in responses. It operates as a data-plane single binary without an Envoy-sidecar, and includes a web console and auditing capabilities.

How it works

Requests pass through a single handler: read client request → mask sensitive values → forward to the upstream provider → receive response and restore originals (entire body or token-by-token for SSE) → return to client. The masking table is kept in memory for the duration of the request. It supports OpenAI and Anthropic endpoints and can be configured via GUARDRAILS_PATHS for custom paths. If the request body cannot be parsed, it proceeds unmasked and increments a counter. The service only replaces values with placeholders and does not block requests.

Getting started

Quick start: run the provided Docker image and point clients to localhost:8080; upstream base URL is set via GUARDRAILS_UPSTREAM_BASE_URL. Example commands:

docker run --rm -p 8080:8080 -p 9080:9080 \
  -e GUARDRAILS_UPSTREAM_BASE_URL=https://foundation-models.api.cloud.ru \
  ghcr.io/cloud-ru-tech/guardrails-llm-filter:latest

curl -sS http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $CLOUDRU_API_KEY" \
  -d '{"model":"ai-sage/GigaChat3-10B-A1.8B","messages":[{"role":"user","content":"email me at a@b.com"}]}'

Web console is available at http://localhost:9080.

Building from source

make build
GUARDRAILS_UPSTREAM_BASE_URL=https://foundation-models.api.cloud.ru \\
  ./bin/guardrails-llm-filter

Quickstart demo

cd examples/quickstart
docker compose up --build
bash demo.sh

The CLOUDRU_API_KEY and model list are documented in Cloud.ru Foundation Models docs.

Recent releases

• v0.1.1 (2026-07-14): redesign of the web console, monitoring page, and diagrams in documentation by @mr8bit; new contributor @mr8bit.
• v0.1.0-alpha (2026-07-13).

Traction

Stars: 31

Configuration

Variables with prefix GUARDRAILS_ (examples):

• GUARDRAILS_UPSTREAM_BASE_URL (обязательно)
• GUARDRAILS_LISTEN_ADDR (default :8080)
• GUARDRAILS_MODE (default enforce; detect = shadow mode)
• GUARDRAILS_STORE_BACKEND (default in_memory; options: redis | postgres)
• GUARDRAILS_AUDIT_ENABLED (default false)
• GUARDRAILS_API_ADDR (default :9080)

Полный набор окружения доступен в разделе Конфигурация с подробностями и полным справочником переменных окружения.