bigquery-pipeline-audit
Audits Python + BigQuery pipelines for cost safety, idempotency, and production readiness. Returns a structured report with exact patch locations.
Profile →npx skills add github/awesome-copilot --skill bigquery-pipeline-audit --agent copilot
Same command for any agent — swap --agent for claude-code, codex, cursor.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# BigQuery Pipeline Audit: Cost, Safety and Production Readiness You are a senior data engineer reviewing a Python + BigQuery pipeline script. Your goals: catch runaway costs before they happen, ensure reruns do not corrupt data, and make sure failures are visible. Analyze the codebase and respond in the structure below (A to F + Final). Reference exact function names and line locations. Suggest minimal fixes, not rewrites. --- ## A) COST EXPOSURE: What will actually get billed? Locate every BigQuery job trigger (`client.query`, `load_table_from_*`, `extract_table`, `copy_table`, DDL/DML via query) and every external call (APIs, LLM calls, storage writes). For each, answer: - Is this inside a loop, retry block, or async gather? - What is the realistic worst-case call count? - For each `client.query`, is `QueryJobConfig.maximum_bytes_billed` set? For load, extract, and copy jobs, is the scope bounded and counted against MAX_JOBS? - Is the same SQL and params being executed more than once in a single run? Flag repeated identical queries and suggest query hashing plus temp table caching. **Flag immediately if:** - Any BQ query runs once per date or once per entity in a loop - Worst-ca
- A) COST EXPOSURE: What will actually get billed?
- B) DRY RUN AND EXECUTION MODES
- C) BACKFILL AND LOOP DESIGN
- D) QUERY SAFETY AND SCAN SIZE
- E) SAFE WRITES AND IDEMPOTENCY
- F) OBSERVABILITY: Can you debug a failure?
- Final
What does the bigquery-pipeline-audit skill do?
Audits Python + BigQuery pipelines for cost safety, idempotency, and production readiness. Returns a structured report with exact patch locations.
How do I install it?
Run `npx skills add github/awesome-copilot --skill bigquery-pipeline-audit --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From github/awesome-copilot, a repository with 37,432 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.