Simple LLM Finetuner provides a Gradio-based UI to fine-tune language models using PEFT/LoRA on consumer GPUs. It includes dataset pasting, adjustable fine-tuning and inference parameters, and saving LoRA adapters to a lora/ directory.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Simple LLM Finetuner is a beginner-friendly interface designed to facilitate fine-tuning various language models using LoRA via the PEFT library on commodity NVIDIA GPUs. With small dataset and sample lengths of 256, you can even run this on a regular Colab Tesla T4 instance.
How it works
The UI lets you paste datasets separated by double blank lines, adjust fine-tuning and inference parameters, and train a LoRA adapter. After training, you can perform inference by selecting the LoRA in the Inference tab.
Getting started
- Prerequisites:
- Linux or WSL
- Modern NVIDIA GPU with >= 16 GB of VRAM
- Create a Python environment and install packages as shown:
conda create -n simple-llm-finetuner python=3.10
conda activate simple-llm-finetuner
conda install -y cuda -c nvidia/label/cuda-11.7.0
conda install -y pytorch=2 pytorch-cuda=11.7 -c pytorch
On WSL, configure CUDA as needed and set LD_LIBRARY_PATH as shown in the README steps.
- Clone and install requirements:
git clone https://github.com/lxe/simple-llm-finetuner.git
cd simple-llm-finetuner
pip install -r requirements.txt
- Launch:
python app.py
Open http://127.0.0.1:7860/ in your browser.
- Data prep: separate each sample with 2 blank lines. Paste into the textbox. Set the New PEFT Adapter Name, then train. Adjust max sequence length and batch size as needed. The model will be saved in the
lora/directory.
After training, use the Inference tab to select your LoRA and test.
Recent releases
- none
Traction
- stars: 2053
Behind the repo
- language: Jupyter Notebook
- license: MIT
- created: 2023-03-22
- last_push: 2023-12-21
- topics: ai, gpt-2, gpt-3, huggingface, huggingface-transformers, llama, llm, peft, pytorch
Caveats
- license: MIT
- last_push: 2023-12-21
- open_issues: 36