Open-source Chinese Llama 2 7B implementation with Docker, GGML/CPU paths, and Python-based utilities. It provides model access, demos, and deployment options across Docker and CPU/ggml workflows.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Chinese Llama 2 7B is an open-source Chinese-language Llama2 model and SFT data set, with inputs formatted for llama-2-chat compatibility and ready for commercial use under Apache-2.0. The project includes online demos, Colab notations for high-RAM usage, and multiple download links for the model and quantized variants.
How it works
The repository provides: (1) Docker-based deployment using a provided Dockerfile to run a Chinese Llama2 chat application, (2) GGML/llama.cpp paths for CPU execution with a conversion workflow and quantization steps, (3) an API deployment option via a simple FastAPI/uvicorn setup with an api.py script for local hosting. It includes example code illustrating how to load the model and run a generation with a TextStreamer to stream outputs.
Getting started
Quickstart paths include:
- Docker: use the Dockerfile to build and run with a command sequence:
docker build -t linksoul/chinese-llama2-chat .
docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --rm -it -v
pwd/LinkSoul:/app/LinkSoul -p 7860:7860 linksoul/chinese-llama2-chat - GGML/CPU: use the provided scripts to convert to ggml and quantize, including a sample command sequence for conversion and quantization.
- API部署: install fastapi uvicorn, then run python api.py and call the local endpoint via curl as shown in the README.
Recent releases
No explicit releases are listed under RELEASES (latest 0) for this repository.
Traction
The repository has 2206 stars and 197 forks as raw counts. It also shows 31 open issues. Language is Python and license is Apache-2.0. Created 2023-07-20, last push 2023-10-26. Topics include deep-learning, llama2, llama2-docker, llm, pytorch.
Caveats
License is Apache-2.0. The README includes multiple Colab links and Colab (INT4/FP16) notes requiring high RAM, indicating resource requirements for running the model. There are no explicit release notes beyond updates listed in 快速更新; latest updates appear dated 10月26日 and earlier. The Docker section references a specific Nvidia PyTorch base image in a suggested workflow and provides commands for building and running the container.