anomaly-detection

What it does

Implements rule-based anomaly detection with cooldowns and error pattern tracking for production systems. It supports detecting slow jobs, high failure rates, unhealthy workers, queue backlogs, repeated errors, and memory spikes, with per-rule cooldowns to prevent alert storms and decay-like behavior upon recovery.

How it works

• TypeScript implementation defines:
  • Enums for AnomalyType and AnomalySeverity.
  • Interfaces for AnomalyAlert, RuleContext, and AnomalyRule.
  • A defined list ANOMALY_RULES containing rules with anomalyType, severity, description, a checkFn, a messageTemplate, and cooldownSeconds.
  • An AnomalyDetector class with internal state (alerts, cooldowns, errorCounts, timeoutCounts, alertIdCounter).
  • checkWorkerHealth(workerName, health) builds a RuleContext and iterates ANOMALY_RULES, skipping on cooldowns, applying rule.checkFn(ctx), creating alerts via createAlert, and setting cooldowns accordingly.
  • checkJobExecution(workerName, jobId, durationMs, expectedDurationMs, success, error) tracks errors, checks for slow jobs, and repeated errors, creating alerts and setting cooldowns when rules fire.
  • resolveAnomaly(alertId, resolution) marks an alert resolved with resolution text.
  • getActiveAnomalies() returns unresolved alerts ordered by severity.
  • Helper methods: trackError, createAlert, isOnCooldown, setCooldown.
• Python implementation mirrors the TS logic:
  • Defines AnomalyType and AnomalySeverity enums, AnomalyAlert and RuleContext dataclasses, AnomalyRule dataclass.
  • Uses a list ANOMALY_RULES with similar rules and cooldowns.
  • AnomalyDetector class provides check_worker_health, check_job_execution, resolve_anomaly, get_active_anomalies, and internal tracking similar to TS, including _track_error, _create_alert, _is_on_cooldown, and _set_cooldown.
  • Timezone-aware timestamps and consistent field naming (snake_case) in dataclasses.

When to use it

• When you need to detect slow job degradation before failures.
• When you want to track error rate creep over time.
• When you want to identify repeated error patterns.
• When you want to prevent alert fatigue with cooldowns.

What it can touch

• The skill uses only internal state and the provided analysis logic; it creates and returns AnomalyAlert objects, tracks cooldowns, and maintains error counts. It does not specify external file I/O or network access within the shown code.

Caveats

• The rules and cooldowns are defined in code (e.g., 300s for SLOW_JOB, 600s for HIGH_FAILURE_RATE).
• The repetition threshold for repeated errors is 5 occurrences, with a 900s cooldown.
• The memory spike rule uses a 1024 MB threshold.
• The Python and TypeScript versions align but may differ in exact type annotations and date handling details (timezone handling via datetime.timezone.utc in Python; Date.now in JS).