web-scraping

npx skills add ginlix-ai/LangAlpha --skill web-scraping --agent claude-code

Same command for any agent — swap --agent for codex, cursor, copilot.

# Web Scraping with Scrapling ## Overview Two ways to scrape in the sandbox: 1. **MCP tool wrappers** (recommended for simple fetches) — call `get()`, `fetch()`, `stealthy_fetch()` directly. Synchronous, returns dicts. 2. **Direct Python API** (for advanced use) — import Scrapling classes for selectors, sessions, spiders. Async, returns Page objects. ## MCP Tool Wrappers (via Python) Auto-registered as top-level functions in the sandbox. No imports needed. **Synchronous** — no `await`. Quick fetches can run inline via `ExecuteCode`. For spiders, multi-URL crawls, or anything you'll iterate on, write the scraper to `work/<task_name>/scraper.py` and run it via `Bash` — edit-and-rerun beats resubmitting code. ### Basic Usage ```python # Fast HTTP fetch → markdown result = get(url="https://example.com", extraction_type="markdown") print(result["status"]) # 200 print(result["url"]) # "https://example.com" print(result["content"][0]) # markdown string (first element of list) # Browser fetch for JS-rendered pages result = fetch(url="https://spa-site.com", extraction_type="markdown", network_idle=True) # Anti-bot bypass (Cloudflare, etc.) result = stealthy_fetch(url="https://protected-site