InternGPT is an open source platform to interact with AI models via pointing gestures and multimodal tools, with online demos and a local run option. It supports DragGAN, ImageBind, HuskyVQA, and interactive image editing through a Gradio-based service.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
InternGPT (iGPT) is an open source demo platform that enables interaction with AI models through clicking, dragging, and drawing using a pointing device. It includes support for DragGAN, ImageBind, multimodal dialogue, and interactive image editing. The project provides an online demo and instructions to run locally.
How it works
The repository describes loading multiple tools/models (e.g., HuskyVQA, SegmentAnything, ImageOCRRecognition, StyleGAN) into a Gradio app via a Python script. A sample command shows starting a Gradio service with a specific load string to initialize a set of components. There is also a fully featured launch that loads a large list of modules for all features:
python -u app.py \
--load "ImageOCRRecognition_cuda:0,Text2Image_cuda:0,SegmentAnything_cuda:0,ActionRecognition_cuda:0,VideoCaption_cuda:0,DenseCaption_cuda:0,ReplaceMaskedAnything_cuda:0,LDMInpainting_cuda:0,SegText2Image_cuda:0,ScribbleText2Image_cuda:0,Image2Scribble_cuda:0,Image2Canny_cuda:0,CannyText2Image_cuda:0,StyleGAN_cuda:0,Anything2Image_cuda:0,HuskyVQA_cuda:0" \
-p 3456 --https -e
There are also instructions for selectively loading features (e.g., for DragGAN only):
python -u app.py --load "StyleGAN_cuda:0" --tab "DragGAN" --port 3456 --https -e
python -u app.py --load "HuskyVQA_cuda:0,SegmentAnything_cuda:0,ImageOCRRecognition_cuda:0" --port 3456 -e
If voice is enabled, an OpenSSL certificate is required and a sample setup is shown:
mkdir certificate
openssl req -x509 -newkey rsa:4096 -keyout certificate/key.pem -out certificate/cert.pem -sha256 -days 365 -nodes
python -u app.py --load "HuskyVQA_cuda:0,SegmentAnything_cuda:0,ImageOCRRecognition_cuda:0" \
--port 3456 --https -e
Getting started
- Run a Gradio service with basic features using:
python -u app.py --load "HuskyVQA_cuda:0,SegmentAnything_cuda:0,ImageOCRRecognition_cuda:0" --port 3456 -e
- For all features, run:
python -u app.py \
--load "ImageOCRRecognition_cuda:0,Text2Image_cuda:0,SegmentAnything_cuda:0,ActionRecognition_cuda:0,VideoCaption_cuda:0,DenseCaption_cuda:0,ReplaceMaskedAnything_cuda:0,LDMIn painting_cuda:0,SegText2Image_cuda:0,ScribbleText2Image_cuda:0,Image2Scribble_cuda:0,Image2Canny_cuda:0,CannyText2Image_cuda:0,StyleGAN_cuda:0,Anything2Image_cuda:0,HuskyVQA_cuda:0" \
-p 3456 --https -e
- To disable unnecessary dependencies and only try DragGAN:
python -u app.py --load "StyleGAN_cuda:0" --tab "DragGAN" --port 3456 --https -e
Getting started (continued)
- If you want the online demo, visit: https://igpt.opengvlab.com
- The README indicates the project is under construction and invites contributions.
Recent releases
- The Releases section lists none for the latest (0).
Traction
- Stars: 3207
License
- Apache-2.0
What it supports (from README)
- DragGAN, ImageBind, HuskyVQA, interactive image editing, interactive image generation, interactive visual question answering, segment anything, image inpainting, image caption, image matting, OCR, action recognition, video caption, video dense caption, video highlight interpretation.