A Swift/Metal course for building a complete LLM inference engine on Apple Silicon, with a studio app, companion book, and 47 problems. 129 stars, 11 forks, Apache-2.0 license.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (2 so far).
What it is
Inference School is a problem-based course that guides learners through building a small language-model inference engine for Apple Silicon. It provides both a Swift CPU implementation and a Metal implementation for each numerical operator, with an emphasis on correctness, performance, and explainability. The project includes a native macOS Studio app, a command-line interface, and a companion book.
How it works
Problems progress from basic operations (e.g., dot product) to full attention mechanisms, with two execution paths for each operator:
- A Swift CPU implementation serving as the correctness oracle
- A Metal implementation focused on memory traffic and dispatch shape Learners predict performance, run experiments, inspect results, and integrate operators into an inference engine. The Studio provides a reader, editable code, checks, and benchmarks; the CLI exposes similar functionality.
Getting started
From the repository root, use the packaging script and open the Studio:
scripts/package-studio.sh debug
open "dist/Inference School Studio.app"
The packaging creates an ad hoc signed, sandboxed app for local use. It opens at the reader-only orientation lesson 000.
Companion book verification and access:
make -C Book check
open "dist/Inference-School-Companion.pdf"
open "dist/Inference-School-Companion.epub"
Verify local tools:
swift --version
xcrun --find metal
Command reference examples:
swift run inference-school list
swift run inference-school learn 001
swift run inference-school show 004
swift run inference-school check 004 --cpu
Recent releases
- None
Traction
129 stars, 11 forks, 1 open issue
Behind the repo
The project is written in Swift and targets Apple Silicon macOS with requirements including macOS 15+, Xcode/Tools, and Swift/Metal availability.
Caveats
License: Apache-2.0. The packaged Studio is signed with App Sandbox and is not notarized for distribution. The repo notes that the book toolchain requires Python 3, Pandoc, LuaLaTeX, latexmk, Ghostscript, and EPUBCheck for verification.