bio-batch-downloads
Download large datasets from NCBI efficiently using EPost, history server, batching, rate limiting, and retry logic. Use when bulk-fetching tens of thousands of sequences, pulling all results of a large ESearch, designing reproducible pipelines, comparing E-utilities to NCBI Datasets v2 CLI, or implementing checksum-validated downloads. Encodes WebEnv TTL (~8h), EPost 200-ID limit, retmax caps, parallelization design, and integrity verification.
npx skills add BioTender-max/awesome-bio-agent-skills --skill batch-downloads --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
## Version Compatibility Reference examples tested with: BioPython 1.83+, NCBI Datasets CLI 16.0+, Entrez Direct 21.0+ Before using code patterns, verify installed versions match. If versions differ: - Python: `pip show biopython` then `help(Bio.Entrez.efetch)` to check signatures - CLI: `datasets --version` and `efetch -version` If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying. # Batch Downloads **"Download N thousand records from NCBI without getting blocked"** -> The right answer is rarely "parallelize requests". For >5000 records the answer is the **history server**: search once, fetch in chunks server-side. For >100,000 records or whole genomes, the modern answer is **NCBI Datasets v2 CLI** -- the E-utilities are not optimized for bulk genome/gene data anymore. This skill encodes (a) when to use each retrieval strategy, (b) the precise rate-limit math, (c) WebEnv lifecycle for long-running jobs, (d) how to design retry/resume, and (e) when to defect to Datasets CLI instead. - Python: `Entrez.esearch(usehistory='y')` + chunked `Entrez.efetch()` (BioPython) - CLI: `datase
- Version Compatibility
- Required Setup
- Decision matrix: which retrieval strategy?
- Rate-limit math (precise)
- History server lifecycle (the long-running-job trap)
- EPost specifics
- Batch size guidelines per rettype
- Code patterns
- Production batch fetch (history server + retry + checkpoint)
- EPost large ID list, then EFetch
- Integrity check after download
- Compare E-utils to Datasets CLI cost
- Parallelization design (modest)
- Failure modes
What does the bio-batch-downloads skill do?
Download large datasets from NCBI efficiently using EPost, history server, batching, rate limiting, and retry logic. Use when bulk-fetching tens of thousands of sequences, pulling all results of a large ESearch, designing reproducible pipelines, comparing E-utilities to NCBI Datasets v2 CLI, or implementing checksum-validated downloads. Encodes WebEnv TTL (~8h), EPost 200-ID limit, retmax caps, parallelization design, and integrity verification.
How do I install it?
Run `npx skills add BioTender-max/awesome-bio-agent-skills --skill batch-downloads --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From BioTender-max/awesome-bio-agent-skills, a repository with 135 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.