bitsandbytes enables 8-bit and 4-bit quantization for PyTorch to reduce memory for LLMs, with CUDA/ROCm/CPU support and documented system requirements. Latest updates as of 2026-07-29 on main branch. Repository has active releases and broad hardware coverage.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
bitsandbytes enables accessible large language models via k-bit quantization for PyTorch. We provide 8-bit optimizers, LLM.int8() 8-bit quantization for inference, and QLoRA 4-bit quantization for training. The library includes quantization primitives such as bitsandbytes.nn.Linear8bitLt, bitsandbytes.nn.Linear4bit, and 8-bit optimizers through the bitsandbytes.optim module.
How it works
The project implements quantization primitives for 8-bit and 4-bit operations to reduce memory usage during inference and training. It supports various accelerator backends and hardware platforms, with architecture-specific optimizations noted in the README's accelerator table. Documentation links reference Hugging Face Transformers, Diffusers, and PEFT integrations for quantization workflows.
Getting started
System requirements (from README):
- Python 3.10+
- PyTorch 2.4+ Accelerator support and platform-specific notes are provided in the README's table and the linked documentation. See the README for full accelerator coverage and compatibility notes.
Recent releases
Latest 5 releases on main:
- 0.50.0 (2026-07-25): Faster 4-bit inference, CPU/ROCm/Apple Silicon improvements, fused 4-bit GEMM for inference on CUDA and ROCm, faster CPU ops on x86-64 and ARM64, reduced host-side overhead, improved Apple Silicon backend.
- 0.49.2 (2026-02-16): Default 64 blocksize for 4bit quantization supported on ROCm; ROCm 7.2 build
- 0.49.1 (2026-01-08): Update AMD targets; guard for self.weight.quant_state attribute
- 0.49.0 (2025-12-11): x86-64 CPU improvements for 4bit on AVX512 variants; AMD ROCm improvements
Traction
Stars: 8387 Forks: 899 Open issues: 54
Behind the repo
Linked to MIT license and active maintenance with continuous releases from main branch. Documentation links point to official docs and Hugging Face integrations.
Caveats
License: MIT Created: 2021-06-04 Last push: 2026-07-29