arize-instrumentation

What it does

Directs an agent to add Arize AX tracing to an LLM application using a two-phase flow: phase 1 analyzes the codebase and phase 2 implements instrumentation after user confirmation. It supports Python, TypeScript/JavaScript, Java, and Go apps, with specific wiring for environments, credentials, and OpenTelemetry integration. It emphasizes inspecting the codebase first, not mutating logic, preferring auto-instrumentation, and creating a centralized instrumentation module that initializes tracing before LLM clients are created.

How it works

• Phase 0: Environment preflight to determine scope, runtime surface, and verification needs; do not run ax installation yet.
• Phase 1: Analysis (read-only): detect language, package manager, LLM providers, frameworks, and existing tracing, identify scope, and decide integration routing. Return a Phase 1 summary; proceed to Phase 2 only with user confirmation.
• Phase 2: Implementation:
  • Fetch and follow installation/instrumentation steps from matched docs.
  • Install packages per language before coding, e.g., Python: pip install arize-otel plus openinference-instrumentation-{name}; TypeScript/JS: install @opentelemetry/sdk-trace-node and @arizeai/openinference-*; Java: OpenTelemetry SDK and openinference-instrumentation-* in build files; Go: required otel packages and explicit exporter wiring with otlptracehttp.WithEndpoint(...) and WithHeaders(...).
  • Ensure credentials: use environment variables for ARIZE_API_KEY and ARIZE_SPACE, check ax profiles show, or guide through ax profiles create and point to Arize settings page for keys.
  • Create a single central instrumentation module (e.g., instrumentation.py/.ts/.go) initialized before LLM client creation.
  • If an existing TracerProvider exists, attach Arize as an additional exporter rather than replacing setup.
• Implementation rules: prefer auto-instrumentation first, respect project conventions, fail gracefully on missing env vars, and ensure correct import order. Enrich traces with manual CHAIN and TOOL spans when tool execution is involved.

When to use it

Use when the user wants to instrument an app from scratch, set up LLM observability, integrate OpenTelemetry or OpenInference, or begin Arize tracing, especially for Python and TypeScript/JavaScript projects. Trigger Phase 1 analysis first and proceed to Phase 2 only after user confirmation.

What it can touch

• Auto-instrumentation surface where available (language-specific tooling).
• Manual instrumentation points for tool/chain spans if the framework instrumentors do not expose internal tool results.
• Centralized instrumentation module files created at implementation time.

Caveats

• Requires environment variables for ARIZE_API_KEY and ARIZE_SPACE; raw credentials must not be embedded in code.
• For Go, OpenTelemetry setup requires Go ≥ 1.23 and explicit attribute setting for project name in resources.
• If a user’s app uses multiple services in a monorepo, scope must be identified to avoid instrumenting the entire repo.
• CLI/script apps should flush exporters before exit to ensure traces are exported.