A Python-based project with a large number of stars (2019) focusing on running a 28.9M parameter LLM on an ESP32-S3 microcontroller using per-layer embeddings. It includes firmware, wiring, and flashing steps in a nested README, with training and experiments in src/ and RESULTS.md.
What it is
A project that demonstrates running a 28.9M parameter language model on an ESP32-S3 microcontroller without sending data to a server. It uses Per-Layer Embeddings to store most of the model in flash and reads only required rows per token.
How it works
The model stores 25M parameters in a flash lookup table and uses a small fast-memory core for computation, with PSRAM serving as output head and working memory. It reports speed around 9.5 tok/s end to end and 9.7 tok/s of pure compute, and the model size is 14.9MB at 4-bit. Connectivity is none; everything runs on the device.
Getting started
The README points to firmware/esp32_llm/README.md for firmware, wiring, and flashing steps. Training and experiments are in src/ and experiments/. The full method and results are in RESULTS.md.
Recent releases
- none
Traction
Stars: 2019; Stars gained in last 24h: 291; Open issues: 7; Forks: 215; Created: 2026-07-23; Last push: 2026-07-26; Language: Python; License: MIT.
Behind the repo
Linked materials reference TinyStories dataset and Google's Per-Layer Embeddings (Gemma 3n and 4).
Caveats
Model will not answer questions, follow instructions, write code, or know facts due to the small reasoning portion and design choice; involves firmware/flash workflow in the repository.