TruLens provides evaluation and tracking for LLM experiments and AI agents, with OpenTelemetry-based tracing and batch/inline evaluation capabilities. It supports multiple providers and frameworks, and has active release history since 2020.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
TruLens is a tool for evaluating and tracking experiments and AI agents. It instruments applications with a decorator, records traces (latency, inputs, outputs, tokens, cost) per step, and offers evaluator components to score and explain decisions. It supports comparing versions and shipping the version that earns the evaluation.
How it works
Key concepts include:
- OpenTelemetry-based tracing across function calls, LLM generations, retrievals, and tool invocations.
- Seven evaluators measuring aspects of agent behavior (LogicalConsistency, ExecutionEfficiency, PlanAdherence, PlanQuality, ToolSelection, ToolCalling, ToolQuality).
- Batch and inline evaluation modes: instrumented tracing during runtime or offline dataset evaluation via a Run API.
- MCP support for capturing tool call details via a dedicated span type.
- A flexible Selector API to target specific span attributes for evaluation.
Getting started
Install instructions in the README include:
pip install trulens
pip install trulens trulens-providers-openai # OpenAI / Azure OpenAI
pip install trulens trulens-providers-litellm # LiteLLM (Anthropic, Cohere, Mistral, …)
pip install trulens trulens-providers-google # Google Gemini
pip install trulens trulens-providers-bedrock # AWS Bedrock
pip install trulens trulens-providers-cortex # Snowflake Cortex
pip install trulens trulens-providers-huggingface # HuggingFace
pip install trulens trulens-providers-langchain # LangChain models
pip install trulens trulens-apps-langchain # LangChain / LangGraph
pip install trulens trulens-apps-llamaindex # LlamaIndex
It also provides quick-start usage via Colab and a quickstart notebook link in the README.
Recent releases
Latest releases include TruLens 2.10.0 (2026-07-28) introducing AlignmentReport in 2.10.0; prior 2.9.0 (2026-07-23) added Jury, CriteriaABTest, ScoreDistributionAnalyzer, GoldenSetGenerator; 2.8.1 (2026-05-14) added GEPA integration and updated default models; 2.8.0 (2026-04-30) added SchemaValidation and testset generation; 2.7.2 (2026-04-09) added Python 3.14 support and a password-related update.
Traction
Stars: 3483 Forks: 317 Open issues: 77
Behind the repo
No founder or company-specific details are provided in the README excerpt.
Caveats
License: MIT Created: 2020-11-02 Last push: 2026-08-04