Firefly is an open-source large-model training tool supporting pretraining, instruction fine-tuning, and DPO for many models; it offers data, templates, and multi-model training options. This review covers its scope, data, and model weights released.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Firefly is an open-source large-model training tool that supports pretraining, instruction fine-tuning, and DPO for a wide range of models. It enables full-parameter training, LoRA, and QLoRA-based efficient training, with configuration-driven model training.
How it works
The project provides templates and training workflows to align with various open-source chat models, supports using Unsloth to accelerate training and reduce memory, and offers datasets for instruction tuning.
Getting started
Install and usage commands are not included in the truncated README snippet provided. The repository page references releases (v0.0.1-alpha) and a changelog:
- Latest release: v0.0.1-alpha
- Full Changelog: https://github.com/yangjianxin1/Firefly/commits/v0.0.1-alpha
For exact installation and usage steps, refer to the repository's README and release notes on GitHub.
Recent releases
- v0.0.1-alpha v0.0.1-alpha (2024-02-03): experimental release
Full Changelog: https://github.com/yangjianxin1/Firefly/commits/v0.0.1-alpha
Traction
- Stars: 6650
- Forks: 584
- Open issues: 211
Behind the repo
- Organization/owner: yangjianxin1
- Language: Python
- License: none listed
Model weights and evaluation results are provided, including Open LLM Leaderboard standings for several variants such as firefly-mixtral-8x7b and firefly-llama-30b, with per-model scores shown in the model evaluation table.
Caveats
- License is not listed in the provided facts.
- Last push: 2024-10-24
- Topics include many model families and training techniques; actual licensing and redistribution terms should be verified in the repository.