agentsop-streaming-output

What it does

Defines an enhancement overlay for streaming the output of long-running LLM/agent runs from the backend, including final chat tokens, intermediate agent steps, and custom progress events. It also selects transport (SSE vs WebSocket) and handles client disconnects, with guidance on safe streaming and demuxing on a single connection.

How it works

• Activation rules specify when streaming is warranted and what surfaces to stream (chat final tokens, agent steps, custom progress).
• Core mental model emphasizes streaming only what the user needs to see across three audiences: chat final tokens, agent step updates, and progress events; tokens, updates, and custom events are demultiplexed on one wire using a multi-mode stream.
• Workflow steps include:
  • Step 1: Confirm streaming is warranted based on waiting user and run length.
  • Step 2: Classify surface to projection (messages, updates, custom, values, or debug).
  • Step 3: Pick transport (SSE for receive-only, WebSocket for bidirectional, or async generator/gRPC for non-HTTP endpoints).
  • Step 4: Mix token and step streams on one connection with explicit tags (messages, updates, custom).
  • Step 5: Decide disconnect policy (cancel vs detach) before shipping.
  • Step 6: Add backpressure and heartbeat (keep-alive ping, buffer handling, flush settings).
• Operational models describe concrete OP-1 to OP-8 actions, such as streaming final tokens, streaming intermediate steps, emitting custom progress, multiplexing modes, choosing transport, handling disconnects, keeping connections alive, and filtering firehose to final answers.

When to use it

Activates when a long-running run is streaming output to a chat or UI, when you need to surface intermediate steps or progress, or when choosing between SSE and WebSocket transports. It should not activate for fast (<1s) completions, batch/offline jobs, or purely frontend animation questions.

What it can touch

Involves multiple components and tools: LangGraph, LangChain, OpenAI/Anthropic streaming APIs, and SSE/WebSocket transports. The skill uses and references graph.astream with stream_mode, and emits events via SSE or WebSocket, tagging events as "messages", "updates", or "custom".

Caveats

Includes guidance on disconnect behavior, backpressure, and heartbeat strategies to avoid zombie runs and maintain UX. It cautions against interleaving untagged semantic streams and emphasizes demux on the client side via named SSE events. References to external standards are included for event streaming semantics.