Needle is a tiny 26M-parameter Simple Attention Network for on-device function calling, with a Python interface, quickstart commands, and finetuning utilities. It ships weights on Cactus and provides a local UI for testing and training.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
Needle is a 26m parameter "Simple Attention Network" designed for function calling on consumer devices and for local finetuning. It runs with a UI for testing and finetuning, and weights are available on Cactus (weights are auto-downloaded during use).
How it works
The model uses a compact transformer-like architecture with an encoder and decoder structure described in the README, including components like ZCRMSNorm, Self Attn, Cross Attn, and an embedding/text query pathway. It is pretrained on 200B tokens on 16 TPU v6e and then post-trained on 2B tokens of a single-shot function call dataset. The project provides tooling to load checkpoints, generate results, and tokenize, as shown in the Python usage example:
from needle import SimpleAttentionNetwork, load_checkpoint, generate, get_tokenizer
params, config = load_checkpoint("checkpoints/needle.pkl")
model = SimpleAttentionNetwork(config)
tokenizer = get_tokenizer()
result = generate(
model, params, tokenizer,
query="What's the weather in San Francisco?",
tools='[{"name":"get_weather","description":"Get current weather for a city.","parameters":{"location":{"type":"string","description":"City name.","required":true}}}]',
stream=False,
)
print(result)
# [{"name":"get_weather","arguments":{"location":"San Francisco"}}]
Getting started
Quickstart instructions are:
git clone https://github.com/cactus-compute/needle.git
cd needle && source ./setup
needle playground
This opens a web UI at http://127.0.0.1:7860 for testing and finetuning. Weights are auto-downloaded.
Usage (Python) example is provided in the README and shows how to load a checkpoint and run a generate call. The Finetuning section includes:
# Playground (generates data via Gemini, trains, evaluates, bundles result)
needle playground
# CLI (auto-downloads weights if not local)
needle finetune data.jsonl
Data format for finetuning is described, including the expected fields in each JSONL line and the tool/answer schemas.
Getting started (continued)
The repository exposes several CLI commands for interaction:
- needle playground
- needle finetune <data.jsonl>
- needle run --query "..." --tools
- needle train
- needle pretrain
- needle eval --checkpoint <path>
- needle tokenize
- needle generate-data
- needle tpu <action>
Recent releases
There are no releases listed as latest 0.
Traction
The repository shows 3355 stars and 258 forks, with 28 open issues. It is written in Python and licensed under MIT.
Caveats
Created 2026-02-24; last push 2026-07-28. There are 28 open issues. The README notes that small models can be finicky and that larger models may have more scope in conversational settings, but does not provide additional technical caveats beyond those statements.