Edit-Banana

What it is

Edit Banana is a Python-based framework that aims to transform static content, such as diagrams, into editable DrawIO XML. It emphasizes high-fidelity reconstruction that preserves diagram layout, colors, and element relationships, leveraging SAM 3 for segmentation and multimodal language models for text extraction.

How it works

The project pipeline follows these stages:

• Input: Image files (PNG/JPG/BMP/TIFF/WebP).
• Segmentation: Uses SAM3 for element segmentation.
• Text Extraction: Local OCR detects text; high-resolution crops are sent to Pix2Text for LaTeX conversion.
• DrawIO XML Generation: Merges SAM3 spatial data with OCR results to produce editable XML.

Key components include local OCR (Tesseract), Pix2Text for formula recognition, and a token-based multi-user setup with a global lock and LRU cache for embeddings across requests.

Getting started

Phase 1: Environment & Base Setup

• Prerequisites include Python 3.10+ and CUDA-capable GPU. Install PyTorch with CUDA support via:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

• Clone and initialize directories:

git clone https://github.com/BIT-DataLab/Edit-Banana.git
cd Edit-Banana
mkdir -p input output sam3_output

Phase 2: Models & Core Dependencies

pip install -r requirements.txt

Install SAM3 and BPE via:

bash scripts/setup_sam3.sh

Verify with:

python -c "from sam3.model_builder import build_sam3_image_model; print('OK')"

Additional model and OCR tooling setup includes tesseract-ocr for OCR and optional PaddleOCR/formula tools as described in the README.

Phase 3: Configuration & Troubleshooting Copy the example config and edit paths:

cp config/config.yaml.example config/config.yaml

Set sam3.checkpoint_path and sam3.bpe_path to your models/ locations.

Usage via CLI:

python main.py -i input/test_diagram.png

The output XML is saved in the output/ directory. For batch processing, run the command without -i.

Optional: test web API

python server_pa.py

Then POST a file to http://localhost:8000/convert or use the API docs at /docs.

Recent releases

There are no releases listed in the repository metadata.

Traction

GitHub statistics present: 5451 stars, 360 forks, 38 open issues. The README indicates ongoing development and a web-based service.

License

Apache License 2.0

Project structure highlights

• config/ for configuration files (copy config.yaml.example)
• flowchart_text/, input/, models/, output/ directories
• main.py for CLI entry, server_pa.py for FastAPI backend
• requirements.txt listing Python dependencies

Development notes

The project integrates SAM3 (SAM 3) for segmentation, Local OCR (Tesseract), Pix2Text for LaTeX, and a DrawIO XML generation step. It supports a registration/credit system for users and a global lock plus LRU cache to manage concurrent GPU access and embeddings persistence.