RadarTopicsBuildersWeeklyReads
Open Source Radar

Topic: tokenizer

← Back to the radar
CategoriesAgentsMCPCoding agentsLLMRAGInferenceVector DBsBrowserWorkflowstopic: tokenizer ✕
SortTrendingMost starsNewest2 repos · page 1/1
rasbt/
LLMs-from-scratch
▲ 487/wk

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

101k15kJupyter Notebook✓ Reviewed
marcelroed/
gigatoken

Language model tokenization at GB/s

3.9k203Rust✓ Reviewed