LLM-Engineers-Handbook

What it is

The repository is the codebase for the LLM Engineer's Handbook, providing end-to-end LLM-based system guidance, data collection, training pipeline, simple RAG, AWS deployment, monitoring, and evaluation framework. The project includes core package llm_engineering with modules for domain, application, model, and infrastructure, plus pipelines, steps, tests, and utilities. It also references cloud services (HuggingFace, Comet ML, Opik, ZenML, AWS, MongoDB, Qdrant) and a set of ZenML YAML configs.

How it works

Codebase organization includes:

• llm_engineering/: core Python package with subfolders domain/, application/, model/, infrastructure/ (external service integrations like AWS, Qdrant, MongoDB)
• pipelines/: ZenML ML pipelines
• steps/: reusable ZenML steps
• tests/: test examples
• tools/: run.py, ml_service.py, rag.py, data_warehouse.py
• configs/: ZenML YAML configurations
• code_snippets/: standalone example code
• config flow follows infrastructure -> model -> application -> domain

The repository relies on Docker for local infra (MongoDB, Qdrant), ZenML for pipelines, and a REST inference service. It emphasizes environment setup via pyenv, Python 3.11, Poetry, and a .env-based credential approach.

Getting started

Installation steps include:

• Clone and enter repo

  git clone https://github.com/PacktPublishing/LLM-Engineers-Handbook.git
  cd LLM-Engineers-Handbook 
  

• Set up Python environment (Python 3.11) via global or pyenv, with commands showing versions and installation steps
• Install dependencies with Poetry, excluding AWS initially:

  poetry env use 3.11
  poetry install --without aws
  poetry run pre-commit install
  

• Activate environment and use Poe the Poet for project commands:

  poetry shell
  

  poetry poe ...
  

• Create and fill a .env file with credentials for OpenAI, HuggingFace, Comet/Opik, MongoDB, Qdrant, and AWS, plus deployment variables (DATABASE_HOST, USE_QDRANT_CLOUD, QDRANT_CLOUD_URL, QDRANT_APIKEY, AWS_REGION, AWS_ACCESS_KEY, AWS_SECRET_KEY).
• Local development setup uses Docker to run MongoDB and Qdrant; start with:

  poetry poe local-infrastructure-up
  

  To stop:

  poetry poe local-infrastructure-down
  

• Start the inference REST API:

  poetry poe run-inference-ml-service
  

Recent releases

• Latest releases section shows: none

Traction

• Stars: 5268

Behind the repo

• The repository is tied to the Packt book and maps to the Amazon Packt product page; it includes links to HuggingFace model, and cloud tooling integration. It mentions integration with HuggingFace, Comet ML, Opik, ZenML, AWS, MongoDB, Qdrant, and GitHub Actions.

Caveats

• License: MIT
• Created: 2024-04-09
• Last push: 2026-04-22
• Open issues: 34
• Local instructions require Docker >= 27.1.1 and Pyenv/Poetry tooling as specified in the installation steps.