A Python-based hands-on course for building an end-to-end LLM inference path on Apple Silicon, using Qwen3 and vLLM-like concepts. The project includes a structured learning path, environment setup steps, and code/tests scaffolding.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
tiny-llm is a hands-on course for systems engineers who want to understand LLM inference end to end. The course builds a path that loads a Qwen3 model, turns tokens into logits, and generates text, aiming to connect equations to memory traffic, kernel occupancy, KV-cache growth, batching, and request scheduling. The learning path spans four weeks, covering topics from Matmul to Text, KV caches, continuous batching, and paging for KV caches. The project emphasizes implementing operators in Python, C++, or Metal rather than calling optimized MLX operations, with MLX serving as correctness oracle and performance baseline.
How it works
The course uses MLX arrays and the MLX extension runtime without high-level neural-network layers. Each operator taught in a chapter is implemented by the student in a chosen language, while MLX provides correctness and performance benchmarks. The material includes a detailed roadmap of chapters and status indicators for code, tests, and docs across weeks 1–4, with emphasis on practical end-to-end inference components such as attention, RoPE, grouped-query attention, RMSNorm, MLP, KV cache, and decoding.
Getting started
Key setup and verification steps shown in the README include:
pdm install -v
pdm run check-installation
pdm run test-refsol -- -- -k week_1
The tiny_llm package contains the exercises, while tiny_llm_ref contains the reference solution used by tests and the benchmark appendix. The book summary and environment setup are linked for further guidance.
Recent releases
Release history: latest release entry shows "none" (no releases listed).
Traction
Stars: 4440. Forks: 354. Open issues: 18. (Note: numbers are taken exactly as shown in the repository facts.)
Behind the repo
No linked startup or company information is present in the provided data.
Caveats
License: Apache-2.0. Created: 2025-04-19. Last push: 2026-08-03. Language: Python. No explicit open issues or runtime caveats are described beyond the repository metadata provided.