blip-2-vision-language
Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.
npx skills add Orchestra-Research/AI-Research-SKILLs --skill blip-2 --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# BLIP-2: Vision-Language Pre-training Comprehensive guide to using Salesforce's BLIP-2 for vision-language tasks with frozen image encoders and large language models. ## When to use BLIP-2 **Use BLIP-2 when:** - Need high-quality image captioning with natural descriptions - Building visual question answering (VQA) systems - Require zero-shot image-text understanding without task-specific training - Want to leverage LLM reasoning for visual tasks - Building multimodal conversational AI - Need image-text retrieval or matching **Key features:** - **Q-Former architecture**: Lightweight query transformer bridges vision and language - **Frozen backbone efficiency**: No need to fine-tune large vision/language models - **Multiple LLM backends**: OPT (2.7B, 6.7B) and FlanT5 (XL, XXL) - **Zero-shot capabilities**: Strong performance without task-specific training - **Efficient training**: Only trains Q-Former (~188M parameters) - **State-of-the-art results**: Beats larger models on VQA benchmarks **Use alternatives instead:** - **LLaVA**: For instruction-following multimodal chat - **InstructBLIP**: For improved instruction-following (BLIP-2 successor) - **GPT-4V/Claude 3**: For production
- When to use BLIP-2
- Quick start
- Installation
- Basic image captioning
- Visual question answering
- Using LAVIS library
- Core concepts
- Architecture overview
- Model variants
- Q-Former components
- Advanced usage
- Batch processing
- Controlling generation
- Memory optimization
HuggingFace Transformers (recommended) pip install transformers accelerate torch Pillow Or LAVIS library (Salesforce official) pip install salesforce-lavis
What does the blip-2-vision-language skill do?
Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.
How do I install it?
Run `npx skills add Orchestra-Research/AI-Research-SKILLs --skill blip-2 --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From Orchestra-Research/AI-Research-SKILLs, a repository with 11,391 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.