py-xiaozhi

What it is

py-xiaozhi is a lightweight, cross-platform multi-modal AI interaction framework built on Python's async architecture. It supports real-time voice streaming, vision-language tasks, and IoT device control. Deployable across Windows, macOS, Linux desktops, and ARM embedded platforms (Raspberry Pi, Horizon Robotics RDK, Jetson Nano).

How it works

Key components include an MCP tool ecosystem ( Modular JSON-RPC 2.0 tools ), WebSocket/MQTT protocol support, plugin architecture, and an event-driven, async design. It uses Opus for audio processing, Sherpa-ONNX offline models for speech recognition, and a wake word system for on-device listening. The project emphasizes dependency injection, configuration management, and extensible plugins for UI, MCP, wake word, and tools.

Getting started

Documentation and setup guidance are included in the repository. Typical development steps shown in the README include:

• Clone the repository
• Install dependencies and run in GUI or CLI modes
• Commands shown for starting in GUI or CLI, and for selecting protocol (WebSocket or MQTT)

Exact commands from README:

git clone https://github.com/huangjunsen0406/py-xiaozhi.git
cd py-xiaozhi

# Base install (CLI / GPIO mode)
uv sync                                    # Recommended (uv users)
# or: pip install -e .                    # pip users

# GUI mode (extra: PySide6 + qasync)
uv sync --extra gui                        # Recommended (uv users)
# or: pip install -e '.[gui]'             # pip users

# Full development environment (GUI + test / packaging tools)
uv sync --extra gui --group dev

# Code formatting
./format_code.sh

# Run program - GUI mode (default; requires gui extra)
python main.py

# Run program - CLI mode (base install is enough)
python main.py --mode cli

# Specify communication protocol
python main.py --protocol websocket  # WebSocket (default)
python main.py --protocol mqtt       # MQTT protocol