web-scraper
Web scraping inteligente multi-estrategia. Extrai dados estruturados de paginas web (tabelas, listas, precos). Paginacao, monitoramento e export CSV/JSON.
Profile →npx skills add sickn33/agentic-awesome-skills --skill web-scraper --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
What it does
Web scraper performs intelligent, multi-strategy web data extraction across pages, modules, and formats. It targets structured data such as tables, lists, pricing, and other content types, with features for pagination, monitoring, and exporting to CSV, JSON, or Markdown tables.
How it works
The skill executes phases in strict order: CLARIFY, RECON, STRATEGY, EXTRACT, TRANSFORM, VALIDATE, FORMAT. If a URL and data target are provided, it proceeds to Phase 2 without further questions. It uses multiple strategies:
- Strategy A (WebFetch with AI Extraction) for static pages and well-structured HTML.
- Strategy B (Browser Automation) for JS-rendered pages, SPAs, and lazy-loaded data, with steps to open a tab, navigate, dismiss banners, read structure, locate elements, and run JavaScript selectors.
- Strategy C (Bash) for API endpoints and downloadable data via curl and jq.
- Strategy E (Structured Data Extraction) to leverage JSON-LD, microdata, or OpenGraph when available. It supports pagination via page numbers, infinite scroll, and load-more buttons, and can merge data across pages.
When to use it
- Use when the user mentions scraper, scraping, extrair dados web, web scraping, or related terms.
- Use when the request relates to web data extraction with a need for structured outputs (CSV/JSON/Markdown).
What it can touch
- Tools listed include claude-code, antigravity, cursor, gemini-cli, codex-cli. The workflow references WebFetch, Browser automation commands, curl, jq, and JavaScript snippets for extraction. It uses URLs, selectors, and data fields as defined by the chosen extraction mode.
Caveats
- The skill relies on phase coordination and may escalate to browser-based extraction if static fetching is insufficient.
- Outputs depend on the target page structure and available data (tables, lists, pricing, etc.).
# Web Scraper ## Overview Web scraping inteligente multi-estrategia. Extrai dados estruturados de paginas web (tabelas, listas, precos). Paginacao, monitoramento e export CSV/JSON. ## When to Use This Skill - When the user mentions "scraper" or related topics - When the user mentions "scraping" or related topics - When the user mentions "extrair dados web" or related topics - When the user mentions "web scraping" or related topics - When the user mentions "raspar dados" or related topics - When the user mentions "coletar dados site" or related topics ## Do Not Use This Skill When - The task is unrelated to web scraper - A simpler, more specific tool can handle the request - The user needs general-purpose assistance without domain expertise ## How It Works Execute phases in strict order. Each phase feeds the next. ``` 1. CLARIFY -> 2. RECON -> 3. STRATEGY -> 4. EXTRACT -> 5. TRANSFORM -> 6. VALIDATE -> 7. FORMAT ``` Never skip Phase 1 or Phase 2. They prevent wasted effort and failed extractions. **Fast path**: If user provides URL + clear data target + the request is simple (single page, one data type), compress Phases 1-3 into a single action: fetch, classify, and extract in one W
- Overview
- When to Use This Skill
- Do Not Use This Skill When
- How It Works
- Capabilities
- Web Scraper
- Phase 1: Clarify
- Required Parameters
- Optional Parameters
- Clarification Rules
- Discovery Mode
- Phase 2: Reconnaissance
- Step 2.1: Initial Fetch
- Step 2.2: Evaluate Fetch Quality
curl -s "API_URL" | jq '[.items[] | {field1: .key1, field2: .key2}]'
curl -s "CSV_URL" -o /tmp/scraped_data.csv
curl -s "XML_URL" | python3 -c "
import xml.etree.ElementTree as ET, json, sys
tree = ET.parse(sys.stdin)What does the web-scraper skill do?
Web scraping inteligente multi-estrategia. Extrai dados estruturados de paginas web (tabelas, listas, precos). Paginacao, monitoramento e export CSV/JSON.
How do I install it?
Run `npx skills add sickn33/agentic-awesome-skills --skill web-scraper --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From sickn33/agentic-awesome-skills, a repository with 44,414 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.