agentsop-llm-artifact-versioning
Enhancement overlay — version the WHOLE deployable LLM-app artifact as one bundle: prompts + compiled programs + model snapshot pins + retrieval config + eval-set version, versioned together so a deploy is reproducible and rollback is atomic. Activate when preparing to deploy an LLM app, when asking "what exactly is running in prod right now?", when a deploy must be reproducible months later, or when an incident needs a clean rollback. The core independently-mutable parts, not one weights file. Do NOT activate for one-off prompt edits with no deploy, for a single-component demo, or where a ven
npx skills add agentsope/SkillAlchemy --skill agentsop-llm-artifact-versioning --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
What it does
Versioned, reproducible LLM artifact overlay. It treats the entire deployable bundle as a single unit and requires pinning and versioning of prompts, compiled programs, model snapshots, retrieval config, and eval-set version so a deployment is reproducible and rollback is atomic. It activates during deployment preparation, inquiries about what runs in prod, reproducibility needs, or incident rollbacks, and it explicitly states that the artifact is a manifest over multiple independently mutable parts, not a single weights file.
How it works
- Enumerate artifact components: list every prompt, system-prompt file, compiled program path/sha, each model call site with its model, retrieval config (index, embed model, top_k, reranker), eval-set version, and framework versions.
- Pin each component: pin dated snapshots at every call site (no aliases), commit prompts to VCS with content sha256, record each call site in the manifest; pin compiled programs with path and sha; move knobs into version-controlled config and reference config_sha; record eval_set_sha with its canonical bytes.
- Bundle + version: write a single manifest.<version>.json referencing all pinned components and assign a monotonic bundle id via git tag deploy/v<n>.
- Tie to eval-set version: record eval_set_sha and scores; regression gate re-runs the exact eval set on the new bundle vs the parent.
- Enable rollback: production reads the bundle tag, not HEAD; rollback re-points to the previous tag to restore prompts, model, config, and index pointer together.
When to use it
Activate when preparing to deploy, when asked what is running in production, when a reproducible deployment is needed months later, or when a clean rollback is required (e.g., incidents, drift, cross-component bundles). Do not activate for one-off prompt edits with no deploy, a single-component demo, or vendor-managed prompts.
What it can touch
The skill instructs to enumerate and pin components: prompts, compiled programs, model snapshots, retrieval config, eval-set, and framework versions. It uses a manifest backed by a git-tagged bundle and requires per-component identifiers like sha256, path, and version tags.
Caveats
It treats the LLM app artifact as a composite bundle rather than a single weights file; aliases for models are avoided in pinning, and a full rollback requires re-pointing to a previous bundle tag. The rollback relies on the manifest and bundle tag to restore the whole combination atomically.
# Versioned, Reproducible LLM Artifact — Overlay SOP > *"Prompts are effectively the weights of an LLM application."* > — DSPy core philosophy [arxiv.org/abs/2310.03714] (R1 §1) > > *"Treat the compiled program as a (program × LM) pair. Changing the LM invalidates the > artifact — recompile."* > — dspy-sop SKILL, Dilemma Case B (R1 §2) This is an **enhancement overlay**, not a framework SOP. It sits on top of whatever stack you use (DSPy, LangChain, raw API) and adds one discipline: **define, pin, and version the entire deployable bundle as a unit.** It is the broad sibling of [[agentsop-per-model-artifacts]] — that skill versions one compiled prompt; this one versions everything that ships together. --- ## 1. 何时激活 (When to activate) Activate when **any** of these appears in the user's intent, codebase, or workflow: | Trigger | Signal | |---|---| | Preparing to deploy | "ship this to prod", a `Dockerfile`/`deploy.yaml`/serving entrypoint wrapping an LLM app, a release checklist | | "What is running in prod?" | Nobody can name the exact prompt text + model snapshot + retriever config currently serving traffic | | Reproducibility need | "reproduce the deploy from last quarter", an au
- 1. 何时激活 (When to activate)
- 2. 核心心智模型 (Core mental model)
- An LLM app artifact ≠ an ML model
- Why "together" is non-negotiable
- The registry analogy and its limit (R1 §7)
- 3. SOP 工作流 (SOP workflow)
- Step 1 — Enumerate artifact components
- Step 2 — Pin each component
- Step 3 — Bundle + version
- Step 4 — Tie to eval-set version
- Step 5 — Enable rollback
- Maintenance loop
- 4. 操作模型 (Trigger / Action / Output / Evidence)
- OP-1 — Artifact manifest
What does the agentsop-llm-artifact-versioning skill do?
Enhancement overlay — version the WHOLE deployable LLM-app artifact as one bundle: prompts + compiled programs + model snapshot pins + retrieval config + eval-set version, versioned together so a deploy is reproducible and rollback is atomic. Activate when preparing to deploy an LLM app, when asking "what exactly is running in prod right now?", when a deploy must be reproducible months later, or when an incident needs a clean rollback. The core independently-mutable parts, not one weights file. Do NOT activate for one-off prompt edits with no deploy, for a single-component demo, or where a ven
How do I install it?
Run `npx skills add agentsope/SkillAlchemy --skill agentsop-llm-artifact-versioning --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From agentsope/SkillAlchemy, a repository with 255 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.