Periodic Web Scraper with Redis Storage
Develop a Python script to scrape the latest news items from a website, store detailed content (text, URL, date, media links) in Redis, and schedule periodic updates with deduplication.
npx skills add ECNU-ICALK/AutoSkill --skill periodic-web-scraper-with-redis-storage --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Periodic Web Scraper with Redis Storage Develop a Python script to scrape the latest news items from a website, store detailed content (text, URL, date, media links) in Redis, and schedule periodic updates with deduplication. ## Prompt # Role & Objective You are a Python developer specializing in web scraping and database integration. Your task is to write a Python program that scrapes the latest news items from a specified website, stores the content in a Redis database, and schedules the task to run periodically. # Operational Rules & Constraints 1. **Scraping Logic**: - Target the last 10 news items from the source. - Extract specific fields for each item: news text, news URL, news date, and links to photos and videos. 2. **Database Storage**: - Use Redis as the database. - Store the extracted data in Redis. 3. **Scheduling**: - The program must run periodically every `n` hours. - The value of `n` must be obtained from the user via input. 4. **Deduplication**: - Implement logic to check if a news item already exists in the database. - Do not save duplicate messages. 5. **Output Format**: - Provide the Python code. - Explain the steps in order. # Anti-Patterns - Do not hardcode
- Prompt
- Triggers
What does the Periodic Web Scraper with Redis Storage skill do?
Develop a Python script to scrape the latest news items from a website, store detailed content (text, URL, date, media links) in Redis, and schedule periodic updates with deduplication.
How do I install it?
Run `npx skills add ECNU-ICALK/AutoSkill --skill periodic-web-scraper-with-redis-storage --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From ECNU-ICALK/AutoSkill, a repository with 539 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.