RAMalama is an open-source tool that uses OCI containers to simplify local AI model serving and inference. It detects host GPUs, pulls accelerator-specific container images, and exposes REST or chatbot interfaces; installation supports macOS, Fedora, PyPI, and Windows with Docker/Podman.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
RamaLama is an open-source tool that simplifies the local use and serving of AI models for inference from any source through the familiar approach of containers. It eliminates the need to configure the host system by pulling a container image specific to the GPUs discovered on the host system, and allows working with various models and platforms. Models are treated like container images and can be interacted with via REST API or as a chatbot.
How it works
On startup RamaLama inspects the system for GPU support and pulls container images specific to the detected GPUs. It uses container engines (Podman or Docker) to run models inside containers with no-network policy by default and supports multiple AI model registries (transports). The default container engine is Podman when both Podman and Docker are installed; the RAMALAMA_CONTAINER_ENGINE environment variable can override this. It exposes models via REST API or chatbot, and runs containers with --network=none and --rm for cleanup.
Getting started
Install options mentioned in README
Install on macOS (Self-Contained Installer) via the .pkg installer from Releases and run the installer command. See the MACOS_INSTALL.md for details.
Install via PyPI
pip install ramalama
Install script (Linux and macOS)
curl -fsSL https://ramalama.ai/install.sh | bash
Install on Windows
pip install ramalama
Requirements:
- Python 3.9 or later
- Docker Desktop or Podman Desktop with WSL2 backend
- For GPU support, see documentation
Uninstall commands (examples)
pip uninstall ramalama
sudo dnf remove ramalama
sudo rm /usr/local/bin/ramalama
Recent releases
- v0.23.0 (2026-06-24): e2e tests/issues adjustments and registry push test fix.
- v0.22.0 (2026-06-05): macOS install instructions improvements and dependency updates.
- v0.21.0 (2026-05-14): Kubernetes YAML resource limit fix and localhost prefixing.
- v0.20.0 (2026-04-27): dependency upgrade and Podman-style manpage preprocess.
- v0.19.0 (2026-04-17): pre-commit cleanup and version subcommand addition.
Traction
2986 stars, 353 forks, 107 open issues. (latest activity on releases shown above)
Behind the repo
Not provided in the material.
Caveats
- License: MIT
- Accelerated images are listed for various accelerators (GGML_VK_VISIBLE_DEVICES, CUDA_VISIBLE_DEVICES, etc.).
- Requires compatible GPUs and container engines; Windows requires WSL2 backend.