kalosm

What it is

Kalosm is an ecosystem of crates that make it easy to develop applications that use local or remote AI models. There are try main projects in this repo:

• Kalosm: A simple interface for pre-trained models in rust
• Fusor: A runtime for quantized ML inference implemented with WGPU

Kalosm supports models across language, audio, and image modalities, including Llama, Mistral, Phi, Whisper, Segment Anything, and Bert. The project emphasizes local-first interfaces for pre-trained models in Rust and provides utilities for transcription, web scraping, and structured generation.

How it works

Kalosm exposes a Rust-based interface (Kalosm) to interact with pre-trained models in various modalities. It includes:

• A simple interface for pre-trained models in Rust
• A runtime for quantized ML inference (Fusor) that uses WGPU and targets various accelerators
• Structured generation capabilities via a parser engine and support for deriving Parse and Schema on Rust types

Example usage shows creating a model (e.g., Llama::phi_3), creating a chat task, and streaming output.

Getting started

From the README:

• Install Rust
• Create a project and add Kalosm as a dependency with features (e.g., --features llama)
• Add tokio with --features full
• Example code demonstrates initializing a model and starting a chat loop

Code excerpts provided include:

cargo new kalosm-hello-world
cd ./kalosm-hello-world

cargo add kalosm --features llama
cargo add tokio --features full

use kalosm::language::*;

let model = Llama::phi_3().await?;
let mut chat = model.chat()
  .with_system_prompt("You are a pirate called Blackbeard");

cargo run --release

Getting started (commands)

Exact commands quoted from README:

cargo new kalosm-hello-world
cd ./kalosm-hello-world

cargo add kalosm --features llama
cargo add tokio --features full

use kalosm::language::*;

let model = Llama::phi_3().await?;
let mut chat = model.chat()
  .with_system_prompt("You are a pirate called Blackbeard");

cargo run --release

Recent releases

Latest releases include:

• kalosm-0.4.0 kalosm-0.4 (2025-02-09): Introducing Kalosm 0.4: A Local-First AI Meta-Framework for Rust
• kalosm-0.3.0 kalosm-0.3.0 (2024-08-21): Kalosm 0.3
• v0.2.0 (2023-09-12): Floneum 0.2 is here! major features including improved web scraping, reading website feeds
• v0.1.0 (2023-07-12): The first pre-release of Floneum

Traction

Not provided in README beyond overall repository statistics, but repo shows 2215 stars and 132 forks. latest activity last push on 2026-08-01.

Behind the repo

The repository encompasses Kalosm and Fusor as sub-crates:

• Kalosm: simple interface for pre-trained models in Rust
• Fusor: WGPU runtime for quantized ML inference, using gguf format and WebGPU

Caveats

• License: Apache-2.0
• Created: 2023-05-24
• Open issues: 53
• Last push: 2026-08-01
• Language: Rust
• Noted early-stage statement for Fusor regarding production readiness: "Fusor is still early in development and is not ready for production use."