agentsop-observability-setup
Enhancement-overlay skill — the DECISION + WIRING layer for LM observability that the single-backend skills [[langsmith]], [[phoenix]], [[mlflow]] do NOT cover. Each of those installs one backend; none of them help you DECIDE which backend fits your stack/scale/budget, nor give you a one-line autolog that turns it on fast. Use when starting any LM project, before the first deploy, or the moment someone asks "why did it do that?" and there are no traces to answer with. The skill picks a backend by stack (LangSmith for LangChain/LangGraph; Phoenix for OSS/local OpenTelemetry; MLflow for ML-shops
npx skills add agentsope/SkillAlchemy --skill agentsop-observability-setup --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
# Observability Setup — Which Backend + One-Line Autolog > *"Instrument before you need it. The cheapest debugging session is a trace you already have."* This is an **ENHANCE overlay**. The local skills [[langsmith]], [[phoenix]] and [[mlflow]] each teach **one** backend deeply. This skill sits one level up: it answers the question those skills cannot — *which one, and how do I turn it on in a single line right now* — then hands off to the chosen backend's own skill for depth. --- ## 1. 何时激活 (When to activate) Activate at one of three moments — earlier is always cheaper: | Trigger | Signal | |---|---| | **Starting any LM project** | First `dspy.LM` / `ChatOpenAI` / `LlamaIndex` / agent graph in the repo, and no tracing wired yet | | **Before first deploy** | About to ship an LM feature to real users with no trace UI — the AP-15 trap (§6) | | **"Why did it do that?" with no traces** | A bug surfaced, you reach for history, and there is nothing recorded — you are debugging blind | | **Multi-component pipeline** | Retriever + reranker + LLM + tools; per-call printing ([[agentsop-prompt-history-inspect]]) is no longer enough | | **Cost / latency regression** | Need aggregate token & la
- 1. 何时激活 (When to activate)
- 2. 核心心智模型 (Core mental model)
- 3. SOP (Standard operating procedure)
- Step 1 — Pick the backend by constraints
- Step 2 — One-line autolog
- Step 3 — Verify traces actually land
- Step 4 — Add eval hooks
- Step 5 — Hand off to the backend skill
- 4. 操作模型 (Operational model — Trigger / Action / Output / Evidence)
- OP-1 · Backend decision table (the core of this skill)
- OP-2 · One-line autolog per backend
- OP-3 · Verify a trace actually landed
- OP-4 · Add eval hooks on top of raw traces
- OP-5 · What to trace (minimum payload)
What does the agentsop-observability-setup skill do?
Enhancement-overlay skill — the DECISION + WIRING layer for LM observability that the single-backend skills [[langsmith]], [[phoenix]], [[mlflow]] do NOT cover. Each of those installs one backend; none of them help you DECIDE which backend fits your stack/scale/budget, nor give you a one-line autolog that turns it on fast. Use when starting any LM project, before the first deploy, or the moment someone asks "why did it do that?" and there are no traces to answer with. The skill picks a backend by stack (LangSmith for LangChain/LangGraph; Phoenix for OSS/local OpenTelemetry; MLflow for ML-shops
How do I install it?
Run `npx skills add agentsope/SkillAlchemy --skill agentsop-observability-setup --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From agentsope/SkillAlchemy, a repository with 255 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.