node-llama-cpp provides Node.js bindings to run llama.cpp locally with TypeScript support, including JSON schema enforcement on generation and multi-GPU backends. It offers pre-built binaries with fallback to building from source via cmake.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Run AI models locally on your machine using Node.js bindings for llama.cpp. It enforces a model to generate output in a parseable format (e.g., JSON) and can follow a JSON schema for responses.
How it works
The project provides pre-built binaries for macOS, Linux and Windows, with a fallback to download a llama.cpp release and build from source with cmake if binaries are not available. It supports GPU backends (Metal, CUDA, Vulkan) and typesafe TypeScript usage. It exposes a getLlama API to load models and create contexts and chat sessions for prompting.
Getting started
Installation:
npm install node-llama-cpp
Usage example:
import {fileURLToPath} from "url";
import path from "path";
import {getLlama, LlamaChatSession} from "node-llama-cpp";
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const llama = await getLlama();
const model = await llama.loadModel({
modelPath: path.join(__dirname, "models", "Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf")
});
const context = await model.createContext();
const session = new LlamaChatSession({
contextSequence: context.getSequence()
});
const q1 = "Hi there, how are you?";
console.log("User: " + q1);
const a1 = await session.prompt(q1);
console.log("AI: " + a1);
Recent releases
Latest versions include v3.19.1 (2026-07-20) and v3.19.0 (2026-06-30), both introducing Gemma 4 features, with further changes through v3.18.1 (2026-03-17) and v3.18.0 (2026-03-15).
Traction
2148 stars, 210 forks, 30 open issues.
Caveats
License is MIT. It relies on pre-built binaries with a fallback to building from source using cmake. If binaries are unavailable for your platform, the behavior is to download a llama.cpp release and build from source unless NODE_LLAMA_CPP_SKIP_DOWNLOAD is set to true.