gcp-batch-inference

npx skills add majiayu000/claude-skill-registry --skill gcp-batch-inference --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# Overview Get asynchronous, high-throughput, and cost-effective inference for your large-scale data processing needs with Gemini's batch inference (formerly known as batch prediction). This guide will walk you through the value of batch inference, how it works, its limitations, and best practices for optimal results. ## Why use batch inference? In many real-world scenarios, you don't need an immediate response from a language model. Instead, you might have a large dataset of prompts that you need to process efficiently and affordably. This is where batch inference shines. ## Key benefits include - **Cost-Effectiveness** Batch processing is offered at a 50% discounted rate compared to real-time inference, making it ideal for large-scale, non-urgent tasks. Implicit caching is enabled by default for Gemini 2.5 Pro, Gemini 2.5 Flash, and Gemini 2.5 Flash-Lite. Implicit caching provides a 90% discount on cached tokens compared to standard input tokens. However, the discounts for cache and batch don't stack. The 90% cache hit discount takes precedence over the batch discount. - **High rate limits** Process hundreds of thousands of requests in a single batch with a higher rate limit comp