Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.
CPU-only, pure-Rust implementation of LiquidAI's LFM2.5-8B-A1B LLM
Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.
