GenieX is an on-device runtime to run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU, via a single SDK with multiple interfaces.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
GenieX is an on-device Gen AI inference runtime for Qualcomm devices. Bring almost any GGUF model from Hugging Face — or a pre-compiled bundle from Qualcomm AI Hub — and run it locally on the Hexagon NPU, Adreno GPU, or CPU in a few lines of code. One C SDK underneath, exposed through a CLI, Python, Kotlin/Java, Docker, and an OpenAI-compatible server.
How it works
GenieX supports two runtimes: llama.cpp (GGUF) and Qualcomm AI Engine Direct (qairt) for NPU acceleration. It dispatches to the llama.cpp runtime over CPU/GPU/Hexagon HTP kernels or to the Qualcomm AI Engine Direct runtime on the NPU. It provides multiple interfaces (CLI, Python bindings, Android SDK, Docker, C/C++ SDK) across Windows, Linux, and Android platforms.
Getting started
Quickstart
CLI
Install
- Windows ARM64 — download the installer, run it, then open a new terminal.
- Linux ARM64 — one line, no
sudo:curl -fsSL https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/install.sh | sh
Run examples
# GGUF from Hugging Face → llama.cpp (NPU / GPU / CPU)
geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf
# Pre-compiled bundle from Qualcomm AI Hub → Qualcomm AI Engine Direct (NPU)
geniex infer ai-hub-models/Qwen2.5-VL-7B-Instruct
# GGUF from Docker Hub → llama.cpp (NPU / GPU / CPU)
geniex infer docker.io/ai/gemma3
Docs for CLI: Install, Quickstart, Command reference.
Python
Install
pip install geniex
Run examples (GGUF):
from geniex import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3.5-2B-GGUF", precision="Q4_0")
# usage...
Run examples (Pre-compiled bundle):
from geniex import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ai-hub-models/Qwen3-4B")
# usage...
Docs for Python: Install, Quickstart, API reference.
OpenAI-compatible server
Install — ships with the CLI install above. Run
geniex pull ai-hub-models/Qwen3-4B-Instruct-2507
geniex serve
Curl example provided to chat completions endpoint at http://127.0.0.1:18181/v1.
Docs: Local server guide.
Android (Kotlin / Java)
Install — add the SDK dependency:
dependencies {
implementation("com.qualcomm.qti:geniex-android:0.3.1")
}
Run — sample app in qualcomm/ai-hub-apps repository.
Docs: Android install, quickstart, API reference.
Docker
Install
docker pull docker.io/qualcomm/geniex:latest
Run — container wraps the CLI, so geniex infer … works as above.
Docs: Docker guide.
C / C++ SDK
Install — link against the single C header sdk/include/geniex.h.
Docs: sdk/README.md, notes/build.md.
Models
GenieX supports two runtimes: llama.cpp (gguf) and Qualcomm AI Engine Direct (qairt).
- Get models: llama.cpp uses Hugging Face GGUF; qairt uses Qualcomm AI Hub bundles.
- Compute units: llama.cpp supports NPU, GPU, CPU; qairt supports NPU only.
- Best for: llama.cpp for bringing your own GGUF; qairt for highest NPU performance.
Note: For llama.cpp, the Q4_0 precision is recommended for Hexagon NPU support. See Models guide for full list and precisions.
Contributing
Contributions welcome. See CONTRIBUTING.md for guidelines and docs links.
Licensing
BSD 3-Clause. See LICENSE and NOTICE.