Firecrawl Automation
Automate web crawling and data extraction with Firecrawl -- scrape pages, crawl sites, extract structured data, batch scrape URLs, and map website structures through the Composio Firecrawl integration.
npx skills add composio-community/awesome-codex-skills --skill firecrawl-automation --agent codex
Same command for any agent — swap --agent for claude-code, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Firecrawl Automation Run **Firecrawl** web crawling and extraction directly from Claude Code. Scrape individual pages, crawl entire sites, extract structured data with AI, batch process URL lists, and map website structures without leaving your terminal. **Toolkit docs:** [composio.dev/toolkits/firecrawl](https://composio.dev/toolkits/firecrawl) --- ## Setup 1. Add the Composio MCP server to your configuration: ``` https://rube.app/mcp ``` 2. Connect your Firecrawl account when prompted. The agent will provide an authentication link. 3. Be mindful of credit consumption -- scope your crawls tightly and test on small URL sets before scaling. --- ## Core Workflows ### 1. Scrape a Single Page Fetch content from a URL in multiple formats with optional browser actions for dynamic pages. **Tool:** `FIRECRAWL_SCRAPE` Key parameters: - `url` (required) -- fully qualified URL to scrape - `formats` -- output formats: `markdown` (default), `html`, `rawHtml`, `links`, `screenshot`, `json` - `onlyMainContent` (default true) -- extract main content only, excluding nav/footer/ads - `waitFor` -- milliseconds to wait for JS rendering (default 0) - `timeout` -- max wait in ms (default 30000) - `act
- Setup
- Core Workflows
- 1. Scrape a Single Page
- 2. Crawl an Entire Site
- 3. Extract Structured Data
- 4. Batch Scrape Multiple URLs
- 5. Map Website Structure
- 6. Monitor and Manage Crawl Jobs
- Known Pitfalls
- Quick Reference
What does the Firecrawl Automation skill do?
Automate web crawling and data extraction with Firecrawl -- scrape pages, crawl sites, extract structured data, batch scrape URLs, and map website structures through the Composio Firecrawl integration.
How do I install it?
Run `npx skills add composio-community/awesome-codex-skills --skill firecrawl-automation --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From composio-community/awesome-codex-skills, a repository with 15,591 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.