YiVal is a Python-based tool for automatic prompt engineering and evaluation in GenAI apps. It provides a data-driven approach to tuning prompts, RAG configurations, and model parameters, with multiple demo notebooks and a pip install option.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
YiVal is described as a tool for automatic prompt engineering and evaluation-centric tuning for GenAI applications, aimed at optimizing prompts, RAG configurations, and model parameters in a data-driven way.
How it works
The repository presents a framework to experiment with automated prompt generation, evaluation of prompts, and integration with various models (e.g., GPT-4) for prompt tuning and dataset generation. It includes demos and notebooks showing usage for different tasks such as headline generation, prompt retrieval, and guardrails evaluation.
Getting started
Installation options include Docker runtimes and Python packaging. Commands from the README include:
docker pull yival/release:latest
docker run --it yival/release:latest
pip install yival
Development setup using Poetry includes:
poetry install --sync
git clone https://github.com/YiVal/YiVal.git
cd YiVal
For running a sample workflow, the README lists steps to set up environment variables and create a configuration file, then execute YiVal:
yival run config_data_generation.yml
export OPENAI_API_KEY=$YOUR_OPENAI_API_KEY
/ YiVal/src/yival
Recent releases
Latest releases include v0.1.0a10 (2023-12-08) with changes to interactive parameters and demo refinements, followed by v0.1.0a9 (2023-11-01) and v0.1.0a8 (2023-10-11) with README updates and a demo video, then v0.1.0a7 (2023-10-03) and v0.1.0a6 (2023-10-02) with notebook additions and an auto_reply evaluator.
Traction
GitHub stars: 2133 Open issues: 18 Forks: 328
Behind the repo
Not provided in the provided material beyond repository ownership (YiVal) and sponsorship mentions.
Caveats
License: Apache-2.0. The README references Docker images and prerequisites such as Python 3.10+ and an OpenAI API key. Last push date is 2024-04-22. Creation date 2023-07-15. The project is Python-based and focused on AI experimentation tooling, with multiple demo notebooks and tutorials.