LLamaSharp is a .NET library to run LLaMA and related models locally, with CPU and GPU backends. It provides integration points and examples, and supports GGUF model format via backends.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
SciSharp/LLamaSharp is a cross-platform C#/.NET library to run LLaMA (and related models like LLaVA) on local devices, with inference that can use CPU or GPU backends. It emphasizes higher-level APIs and RAG support for embedding LLMs in applications.
How it works
The project relies on native backends compiled from llama.cpp to perform inference, exposing backends for Windows, Linux, and Mac with CPU, CUDA, Metal, and Vulkan options. The README shows a typical usage pattern that loads model weights, creates a context, and runs chat interactions through an InteractiveExecutor with streaming support. The library offers optional integrations with semantic-kernel and kernel-memory, and provides various example apps and demos.
Getting started
Installation steps from the README:
PM> Install-Package LLamaSharp
Then install one or more backends, for example:
- LLamaSharp.Backend.Cpu
- LLamaSharp.Backend.Cuda11
- LLamaSharp.Backend.Cuda12
- LLamaSharp.Backend.Vulkan
Optional for extended functionality:
LLamaSharp.semantic-kernel
LLamaSharp.kernel-memory
Model preparation notes describe using GGUF format files converted from PyTorch or HuggingFace formats and reference conversion steps in llama.cpp readme.
Example usage snippet (from the README) shows:
string modelPath = @"<Your Model Path>";
var parameters = new ModelParams(modelPath)
{
ContextSize = 1024,
GpuLayerCount = 5
};
using var model = LLamaWeights.LoadFromFile(parameters);
using var context = model.CreateContext(parameters);
var executor = new InteractiveExecutor(context);
Getting started (continued)
The README includes an in-depth, runnable chat example in C# demonstrating a chat session with a model, including chat history management and streaming text output.
Recent releases
Latest releases listed:
- v0.27.0 (2026-04-26): Windows arm64 CPU support; update for Qwen3.5/Gemma4 support.
- v0.26.0 (2026-02-15): Add support for Gemma 3n; MTMD implementation.
- v0.25.0 (2025-08-16): Binary updates; API references docs.
- v0.24.0 (2025-05-14): May binary update; Android support.
- v0.22.0 (2025-03-20): Grammar resampling; strict parameter for template retrieval.
Traction
Stars: 3771 Forks: 499 Open issues: 18
Behind the repo
Linked integrations include semantic-kernel and kernel-memory developed in LLamaSharp repository, and external integrations with BotSharp, LangChain, and MaIN.NET.
Caveats
License: MIT Created: 2023-05-09 Last push: 2026-08-01 Age and licensing details: MIT license and activity through 2026-08-01 are indicated. No explicit caveats beyond general notes on model formats (GGUF) and backend compatibility are provided in the excerpt.