LlamaGPT is a self-hosted, offline chatbot powered by Llama 2 with Code Llama support, available under MIT and requiring Docker or local execution. It includes multi-model support, OpenAI-like API, and various installation options.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
LlamaGPT is a self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device. New: Support for Code Llama models and Nvidia GPUs.
How it works
The project provides scripts and deployment options to run LlamaGPT locally or in Kubernetes, exposing an OpenAI-compatible API at http://localhost:3001 and a UI at http://localhost:3000. It supports multiple models (Llama 2 and Code Llama variants) with GPU acceleration via CUDA when available.
Getting started
Installation options include:
-
On UmbrelOS home server (one-click from Umbrel App Store).
-
On M1/M2 Mac:
git clone https://github.com/getumbrel/llama-gpt.git
cd llama-gpt
./run-mac.sh --model 7b
Access at http://localhost:3000. To run 13B or 70B models, use 13b or 70b. For Code Llama models, use code-7b, code-13b, or code-34b.
- Anywhere else with Docker:
git clone https://github.com/getumbrel/llama-gpt.git
cd llama-gpt
./run.sh --model 7b
./run.sh --model 7b --with-cuda
Access at http://localhost:3000.
- Kubernetes:
kubectl create ns llama
kubectl apply -k deploy/kubernetes/. -n llama
OpenAI-compatible API is available at http://localhost:3001/docs.
Recent releases
Releases: latest 0: none
Traction
Stars: 10944, Forks: 703 (data from repo stats)
Behind the repo
No startup/company link provided in the README excerpt.
Caveats
License: MIT. Created 2023-07-22. Last push 2024-04-23. Open issues: 96.