data-engineering
Data engineering patterns for ETL pipelines, data warehousing, Apache Spark, and data quality validation
Profile →npx skills add rohitg00/awesome-claude-code-toolkit --skill data-engineering --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Data Engineering ## ETL Pipeline Pattern ```python from datetime import datetime from dataclasses import dataclass @dataclass class PipelineResult: records_extracted: int records_transformed: int records_loaded: int errors: list[str] duration_seconds: float class OrderPipeline: def __init__(self, source_db, warehouse_db): self.source = source_db self.warehouse = warehouse_db def extract(self, since: datetime) -> list[dict]: query = """ SELECT o.*, c.name as customer_name, c.segment FROM orders o JOIN customers c ON o.customer_id = c.id WHERE o.updated_at > %s """ return self.source.fetch_all(query, [since]) def transform(self, records: list[dict]) -> list[dict]: transformed = [] for record in records: transformed.append({ "order_id": record["id"], "customer_name": record["customer_name"], "segment": record["segment"].upper(), "total_amount": float(record["total"]), "order_date": record["created_at"].date(), "fiscal_quarter": get_fiscal_quarter(record["created_at"]), "is_high_value": float(record["total"]) > 1000, "loaded_at": datetime.utcnow(), }) return transformed def load(self, records: list[dict]) -> int: return self.warehouse.upsert_batch( table="fact_orders", records=record
- ETL Pipeline Pattern
- Apache Spark Processing
- Data Quality Checks
- Data Warehouse Schema (Star Schema)
- Anti-Patterns
- Checklist
What does the data-engineering skill do?
Data engineering patterns for ETL pipelines, data warehousing, Apache Spark, and data quality validation
How do I install it?
Run `npx skills add rohitg00/awesome-claude-code-toolkit --skill data-engineering --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From rohitg00/awesome-claude-code-toolkit, a repository with 2,438 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.