shepherd

What it does

Tends long-running pipelines (snakemake, nextflow) using a sense-think-act loop. Detects errors, recalls fixes from memory, restarts automatically, checkpoints progress.

How it works

• Initialize: loads tools, checks for an existing task, determines whether to use an isolated session, creates a named pane, launches the pipeline in that pane, tails the pane, and starts a long-task record for tracking.
• Sense: reads new output from the pipeline pane, checks for idle state, searches for error patterns, and returns new_output, is_idle, errors, and a timestamp.
• Think: analyzes sensed data against known habits, checks for completion, attempts to recall fixes for errors, may apply a fix if available, otherwise restarts (up to a max) or escalates on repeated failures, or signals a stall.
• Act: executes the chosen decision, including completing a task, restarting the pipeline, applying a memory-based fix, escalating on errors, or checkpointing on stalls. Updates long-task records and optionally destroys isolated sessions on completion.
• Pattern libraries: provides detection and actions for common Snakemake and Nextflow patterns, linking specific errors to appropriate recovery steps.
• Resume protocol: supports resuming from a checkpoint by loading a snapshot, restoring session and pane states, and deciding whether to resume monitoring or restart the pipeline.

When to use it

Use when you need automated supervision of Snakemake/Nextflow pipelines, with automatic error handling, fixes from memory, and optional notifications. It supports starting in a dedicated session or using the current session.

What it can touch

• Zellij tools: create_named_pane, write_to_pane, tail_pane, wait_for_idle, search_pane, send_keys, read_pane
• Chitta tools: long_task_start, long_task_active, long_task_snapshot, long_task_event, long_task_complete, habit_match, recall, health_check, habit_strengthen

Caveats

• AUTO_FIX and NOTIFY are configurable flags in initialization; behavior relies on memory recalls and available fixes.
• Uses session isolation heuristics to decide whether to run in a dedicated session or current session.
• Escalation occurs after exceeding max restarts per configuration.
• Requires the underlying tools be present and accessible in the runtime environment.