The Alignment Handbook provides recipes and scripts for continued pretraining, instruction tuning, and preference alignment of language models using RLHF methods, with a structure of scripts and recipes for training workflows.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
The Alignment Handbook offers robust training recipes to continue pretraining and to align language models with human and AI preferences. The project organizes its work around scripts and recipes to reproduce models and training runs related to continued pretraining, supervised fine-tuning, and reward modeling methods such as DPO and ORPO.
How it works
The repository contains:
- scripts to train and evaluate models, including steps for continued pretraining, SFT for chat, preference alignment with DPO, and SFT with ORPO, with distributed training support via DeepSpeed ZeRO-3 or LoRA/QLoRA.
- recipes in ./recipes/ that define training runs in YAML format, including models like Zephyr-7b and related training configurations.
- a simple project structure with directories for scripts and recipes used to reproduce models and training runs.
The installation instructions specify a Python environment setup using uv, installation of PyTorch 2.6.0, and additional dependencies, including Flash Attention 2, with commands quoted exactly as in the README.
Getting started
Installation instructions in the README include:
uv venv handbook --python 3.11 && source handbook/bin/activate && uv pip install --upgrade pip
uv pip install torch==2.6.0 --index-url https://download.pytorch.org/whl/cu126
uv pip install .
uv pip install "flash-attn==2.7.4.post1" --no-build-isolation
huggingface-cli login
sudo apt-get install git-lfs
Next, it notes you can inspect scripts and recipes for training instructions.
Recent releases
No releases are listed as latest; the README does reference various historical recipe and model trainings, including Zephyr and SmolLM workflows, but no numbered releases are specified in the provided data.
Traction
5653 stars, 491 forks, 99 open issues.
Behind the repo
Not provided in facts.
Caveats
License: Apache-2.0. Created: 2023-08-25. Last push: 2026-05-26.