firecrawl-scraping
Web page and website scraping with Firecrawl API. Use this skill when scraping web articles, blog posts, documentation pages, paywalled content, or JavaScript-heavy sites. Triggers on requests to scrape websites, extract article content, convert pages to markdown, or handle anti-bot protection.
npx skills add majiayu000/claude-skill-registry --skill firecrawl-scraping --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Firecrawl Scraping ## Overview Scrape individual web pages and convert them to clean, LLM-ready markdown. Handles JavaScript rendering, anti-bot protection, and dynamic content. ## Quick Decision Tree ``` What are you scraping? │ ├── Single page (article, blog, docs) │ └── references/single-page.md │ └── Script: scripts/firecrawl_scrape.py │ └── Entire website (multiple pages, crawling) └── references/website-crawler.md └── (Use Apify Website Content Crawler for multi-page) ``` ## Environment Setup ```bash # Required in .env FIRECRAWL_API_KEY=fc-your-api-key-here ``` Get your API key: https://firecrawl.dev/app/api-keys ## Common Usage ### Simple Scrape ```bash python scripts/firecrawl_scrape.py "https://example.com/article" ``` ### With Options ```bash python scripts/firecrawl_scrape.py "https://wsj.com/article" \ --proxy stealth \ --format markdown summary \ --timeout 60000 ``` ## Proxy Modes | Mode | Use Case | |------|----------| | `basic` | Standard sites, fastest | | `stealth` | Anti-bot protection, premium content (WSJ, NYT) | | `auto` | Let Firecrawl decide (recommended) | ## Output Formats - `markdown` - Clean markdown content (default) - `html` - Raw HTML - `summary` - A
- Overview
- Quick Decision Tree
- Environment Setup
- Common Usage
- Simple Scrape
- With Options
- Proxy Modes
- Output Formats
- Cost
- Security Notes
- Credential Handling
- Data Privacy
- Access Scopes
- Compliance Considerations
Required in .env python scripts/firecrawl_scrape.py "https://example.com/article" python scripts/firecrawl_scrape.py "https://wsj.com/article" \
What does the firecrawl-scraping skill do?
Web page and website scraping with Firecrawl API. Use this skill when scraping web articles, blog posts, documentation pages, paywalled content, or JavaScript-heavy sites. Triggers on requests to scrape websites, extract article content, convert pages to markdown, or handle anti-bot protection.
How do I install it?
Run `npx skills add majiayu000/claude-skill-registry --skill firecrawl-scraping --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From majiayu000/claude-skill-registry, a repository with 534 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.