EasyR1 is a multi-modality RL training framework for veRL, supporting various LLMs and vision-language models, with Docker and Apptainer deployment options and multiple training scripts.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
EasyR1 is an Efficient, Scalable, Multi-Modality RL Training Framework based on veRL.
How it works
It provides support for multiple models (Llama3, Qwen2/Qwen3 families, Qwen2-VL/VL variants, and DeepSeek-R1 distillates) and several RL algorithms (GRPO, DAPO, Reinforce++, ReMax, RLOO, GSPO, CISPO). It enables features like padding-free training, LoRA training, and checkpoint resuming, with tracking via Wandb, SwanLab, Mlflow, and Tensorboard. It relies on HybirdEngine design and vLLM in SPMD mode for scalability.
Getting started
Installation is shown by the repository:
git clone https://github.com/hiyouga/EasyR1.git
cd EasyR1
pip install -e .
Use the provided scripts to run training, for example:
bash examples/qwen2_5_vl_7b_geo3k_grpo.sh
bash examples/qwen3_vl_4b_geo3k_grpo_lora.sh
python3 scripts/model_merger.py --local_dir checkpoints/easy_r1/exp_name/global_step_1/actor
If there are issues connecting to Hugging Face, set:
export HF_ENDPOINT=https://hf-mirror.com
Docker and environments
docker pull hiyouga/verl:ngc-th2.8.0-cu12.9-vllm0.11.0
docker run -it --ipc=host --gpus=all hiyouga/verl:ngc-th2.8.0-cu12.9-vllm0.11.0
apptainer pull easyr1.sif docker://hiyouga/verl:ngc-th2.8.0-cu12.9-vllm0.11.0
Use USE_MODELSCOPE_HUB=1 to download models from the ModelScope hub.
Hardware and software requirements
Python 3.9+
transformers>=4.54.0
flash-attn>=2.4.3
vllm>=0.8.3
Recent releases
- v0.3.2 (2025-09-18): RL Baselines; fixes include dev versioning and a typo in the tensorboard logger.
- v0.3.1 (2025-06-19): Multi-modal DAPO; updates to runtime environment and references to community examples.
- v0.3.0 (2025-04-15): Initial release; readme and worker fixes for small models.
Traction
5099 stars; 383 forks; 56 open issues.
Caveats
License: Apache-2.0. Created 2025-02-22. Last push 2026-07-30. Hardware guidance provided (estimates) for different bit-widths and model sizes. Known bugs mention vision-language models not compatible with ulysses parallelism yet.