Evidently is an open-source Python library to evaluate, test, and monitor ML and LLM systems, with 100+ built-in metrics and a modular architecture for reports and monitoring.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Evidently is an open-source Python library to evaluate, test, and monitor ML and LLM systems—from experiments to production. It supports tabular and text data, evals for predictive and generative tasks, and provides 100+ built-in metrics. It offers a Python interface for custom metrics and supports both offline evals and live monitoring. The project is modular, enabling starting with one-off evaluations or hosting a monitoring service.
How it works
The library provides:
- Reports and Test Suites that compute and summarize data, ML, and LLM quality evals. Reports can be viewed in Python or exported as JSON, Python dictionary, HTML, or viewed in a monitoring UI. A Report can be converted into a Test Suite with pass/fail conditions.
- A Monitoring Dashboard that visualizes metrics and test results over time via a self-hosted OSS UI or using Evidently Cloud for additional features.
- Descriptors and Presets to configure evaluations (e.g., Data Drift, LLM evals), with examples showing how to construct datasets and evaluators.
Getting started
Install Evidently from PyPI:
pip install evidently
Or install via Conda:
conda install -c conda-forge evidently
Usage examples include importing Report and presets, creating a toy dataset, applying descriptors (e.g., Sentiment, TextLength, Contains), and running a simple TextEvals report. Example snippets are provided for tabular data drift and LLM evaluation, and there is guidance for saving HTML output and obtaining JSON/dict representations of results.
Getting started (continued)
For the monitoring UI, you can run Evidently UI either with uv if available or by installing a virtual environment and running:
evidently ui --demo-projects all
Visit localhost:8000 to access the UI.
Recent releases
Latest releases include v0.7.21 (2026-03-10) with UI logo display in API docs and pagination fix for the reports table; v0.7.20 (2026-01-09) with a docker build fix; v0.7.19 (2026-01-05) with API reference generation fixes; v0.7.18 (2025-12-22) with versioned artefacts and dependency updates; v0.7.17 (2025-11-25) with demo reviews and dataset updates.
Traction
Evidently has 7783 stars and 893 forks on GitHub as of the provided data.
Caveats
The repository is licensed under Apache-2.0 and was created in 2020-11-25. The last push was 2026-05-02. There are 293 open issues. The primary language listed is Jupyter Notebook, indicating extensive notebook-based usage and examples in the project.