asyncopenai-concurrency-httpx-pool
Raise real concurrency in asyncio LLM batch scorers built on the OpenAI SDK (AsyncOpenAI, including OpenAI-compatible providers like DeepSeek). Use when: (1) raising an asyncio.Semaphore above ~100 produces no throughput gain, (2) a batch pipeline saturates near 100 in-flight requests despite a larger semaphore, (3) planning a high-concurrency campaign against a provider with no hard rate limit (DeepSeek v4-flash tolerates 2000+ in flight). Root cause: AsyncOpenAI's default httpx pool caps max_connections at 100, silently bottlenecking any larger semaphore — you must pass a custom http_client
npx skills add kennethkhoocy/applied-micro-skills --skill asyncopenai-concurrency-httpx-pool --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# AsyncOpenAI Concurrency: the Hidden httpx Pool Cap ## Problem Async batch scorers typically gate concurrency with `asyncio.Semaphore(N)`. Raising N above ~100 silently does nothing: the OpenAI SDK's default httpx transport caps the connection pool at `max_connections=100`, so excess tasks queue inside httpx instead of reaching the provider. The semaphore looks like the throttle but is not the binding constraint — there is no error, just a throughput ceiling. ## Context / Trigger Conditions - `asyncio.Semaphore(N)` with N > 100 around `client.chat.completions.create` shows the same throughput as N = 100 - Client constructed as `AsyncOpenAI(api_key=..., base_url=...)` with no `http_client` argument (the default transport) - Provider is known to allow high concurrency (DeepSeek v4-flash: ~2500) - Symptom check: requests-in-flight measured at the server never exceeds ~100 ## Solution Size the httpx pool to the semaphore when constructing the client: ```python import httpx from openai import AsyncOpenAI CONCURRENCY = 2000 client = AsyncOpenAI( api_key=..., base_url="https://api.deepseek.com", http_client=httpx.AsyncClient(limits=httpx.Limits( max_connections=CONCURRENCY, max_keepalive
- Problem
- Context / Trigger Conditions
- Solution
- Verification
- Example
- Notes
What does the asyncopenai-concurrency-httpx-pool skill do?
Raise real concurrency in asyncio LLM batch scorers built on the OpenAI SDK (AsyncOpenAI, including OpenAI-compatible providers like DeepSeek). Use when: (1) raising an asyncio.Semaphore above ~100 produces no throughput gain, (2) a batch pipeline saturates near 100 in-flight requests despite a larger semaphore, (3) planning a high-concurrency campaign against a provider with no hard rate limit (DeepSeek v4-flash tolerates 2000+ in flight). Root cause: AsyncOpenAI's default httpx pool caps max_connections at 100, silently bottlenecking any larger semaphore — you must pass a custom http_client
How do I install it?
Run `npx skills add kennethkhoocy/applied-micro-skills --skill asyncopenai-concurrency-httpx-pool --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From kennethkhoocy/applied-micro-skills, a repository with 54 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.