nanogpt

npx skills add Orchestra-Research/AI-Research-SKILLs --skill nanogpt --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# nanoGPT - Minimalist GPT Training ## Quick start nanoGPT is a simplified GPT implementation designed for learning and experimentation. **Installation**: ```bash pip install torch numpy transformers datasets tiktoken wandb tqdm ``` **Train on Shakespeare** (CPU-friendly): ```bash # Prepare data python data/shakespeare_char/prepare.py # Train (5 minutes on CPU) python train.py config/train_shakespeare_char.py # Generate text python sample.py --out_dir=out-shakespeare-char ``` **Output**: ``` ROMEO: What say'st thou? Shall I speak, and be a man? JULIET: I am afeard, and yet I'll speak; for thou art One that hath been a man, and yet I know not What thou art. ``` ## Common workflows ### Workflow 1: Character-level Shakespeare **Complete training pipeline**: ```bash # Step 1: Prepare data (creates train.bin, val.bin) python data/shakespeare_char/prepare.py # Step 2: Train small model python train.py config/train_shakespeare_char.py # Step 3: Generate text python sample.py --out_dir=out-shakespeare-char ``` **Config** (`config/train_shakespeare_char.py`): ```python # Model config n_layer = 6 # 6 transformer layers n_head = 6 # 6 attention heads n_embd = 384 # 384-dim embeddings block_si

pip install torch numpy transformers datasets tiktoken wandb tqdm
Prepare data
python data/shakespeare_char/prepare.py
Train (5 minutes on CPU)
python train.py config/train_shakespeare_char.py
Generate text
python sample.py --out_dir=out-shakespeare-char
Step 1: Prepare data (creates train.bin, val.bin)
Step 2: Train small model
Step 3: Generate text