agentsop-code-execution-decision
Decision rubric for when an LM agent should write-and-run code (Program-of-Thought / code interpreter) versus reason in natural language: classify each step as deterministic- computable (emit + execute code, feed the result back) vs judgment (stay in prose). Use when designing or debugging an agent step that does arithmetic/parsing/data transforms, when prose reasoning hallucinates a computation (under-coding), or when a sandbox round- trip is wasted on a judgment task (over-coding). Search keywords: code interpreter, agent does math wrong, calculator hallucination, when to run code vs reason,
npx skills add agentsope/SkillAlchemy --skill agentsop-code-execution-decision --agent claude-code
Same command for any agent — swap --agent for codex, cursor, copilot.
Weekly change comes from our own snapshots, not the repository page — it measures attention, not adoption.
What it does
Guides an LM agent to decide, for each step, whether to emit and run code (Program-of-Thought) or to reason in natural language, based on whether the step has a verifiable, deterministic core. It targets arithmetic, parsing, data transformations, or cases where prose would produce unreliable results, and provides a structured gate to choose code when determinism is required.
How it works
- Classifies steps as computable or judgment. If there is a single correct, verifiable answer, route to code execution; otherwise stay in prose.
- If computable, emit a minimal, side-effect-free program (Python is default) and run it in a sandbox, then feed the result back into the LM for narration or chaining.
- If an error occurs, retry up to max_iters (≈ 3); after that, fall back to prose.
- Provides a three-step SOP workflow with concrete actions: decide determinism, emit and run code, then reintegrate results or switch to prose for judgment.
- Includes seven reusable operations addressing gate behavior, decomposing mixed steps, sandbox choice, feeding results, bounded error retry, precision escalation, and vetoing over-coding for judgment tasks.
When to use it
Use before committing a step to a reasoning strategy whenever the task has a verifiable, deterministic core or when a model demonstrates arithmetic in prose. Trigger examples include arithmetic, precise data manipulation, deterministic transforms, and symbolic problems. Also applies to mixed steps where a total is computable but a summary is judgment, guiding decomposition.
What it can touch
- Tools: claude-code is listed as a declared tool.
- It specifies using a sandbox (DSPy Python sandbox, OpenAI Code Interpreter, Anthropic code-execution, LangChain PythonREPLTool) to execute code and feed results back to the LM.
Caveats
- The rubric cautions against using code for purely judgment tasks (tone, summarization, open-ended design) to avoid unnecessary sandboxing.
- It emphasizes that code results should be consumed by the LM rather than replacing the LM, and it caps retries.
- The approach is an enhancement overlay on top of a broader PoT mechanism and references external modules and literature for justification.
# Code-Execution Decision — emit-code-vs-reason > **One-liner**: LMs are unreliable calculators but reliable coders. When the answer needs > determinism and precision — arithmetic, exact data manipulation, deterministic transforms — > emit code and run it. When the answer needs judgment, taste, or open-ended synthesis, reason > in natural language. The cost of getting this gate wrong is silent: prose arithmetic > *hallucinates a plausible-looking wrong number*, and over-coding a judgment task burns a > sandbox round-trip for nothing. This is an **enhancement overlay**. DSPy already gives you `dspy.ProgramOfThought` (PoT) — the *mechanism* for write-then-execute. What it does not give you is the *decision rubric* for when to reach for it. That rubric is this skill. Cross-link the sibling [[agentsop-output-format-by-model]] (which decides *how* code-shaped content should be serialized) and [[agentsop-test-fix-loop]] (which closes the execute → error → retry loop). --- ## 1. 何时激活 (When to activate) Activate this skill **before** committing a step to a reasoning strategy whenever the task has a **verifiable, deterministic core** — or whenever you catch an agent doing arithmetic in pros
- 1. 何时激活 (When to activate)
- 2. 核心心智模型 (Core mental model)
- 3. SOP 工作流 (Decision workflow)
- Step 1 — Classify the step: deterministic-computable vs judgment
- Step 2 — If computable, emit + run code
- Step 3 — Sandbox → run → feed result back into the LM
- 4. 操作模型 (Operations: Trigger → Action → Output → Evidence)
- 5. 困境决策案例 (Dilemma cases)
- Case A — Arithmetic in prose hallucinates → PoT fixes it (under-coding)
- Case B — Over-coding a judgment task wastes a sandbox round-trip (over-coding)
- Case C — Mixed step: summary + total (decompose)
- 6. 反模式与边界 (Anti-patterns & boundaries)
- Anti-patterns
- Boundaries (when this gate doesn't fire)
What does the agentsop-code-execution-decision skill do?
Decision rubric for when an LM agent should write-and-run code (Program-of-Thought / code interpreter) versus reason in natural language: classify each step as deterministic- computable (emit + execute code, feed the result back) vs judgment (stay in prose). Use when designing or debugging an agent step that does arithmetic/parsing/data transforms, when prose reasoning hallucinates a computation (under-coding), or when a sandbox round- trip is wasted on a judgment task (over-coding). Search keywords: code interpreter, agent does math wrong, calculator hallucination, when to run code vs reason,
How do I install it?
Run `npx skills add agentsope/SkillAlchemy --skill agentsop-code-execution-decision --agent claude-code` — it drops the skill into your project so the agent can pick it up. Swap the --agent value for codex, cursor or copilot if you use one of those.
Where does this skill come from?
From agentsope/SkillAlchemy, a repository with 255 stars. We read it straight from the repository tree rather than a submitted listing, so what you see here is what is actually published.
Is a popular skill a good skill?
Not necessarily. Stars measure attention, not adoption — a repository can trend for a week and be abandoned. That is why we show the weekly change from our own snapshots next to the total, instead of a single flattering number.