api-ai-langfuse

What it does

Guides adding observability to LLM applications using Langfuse. It instructs installing tracing and OpenTelemetry packages, enabling automatic context propagation via startActiveObservation or observe, and using observeOpenAI for OpenAI SDK tracing. It also describes managing prompts with LangfuseClient, applying scores/evaluations, and using datasets/experiments. It emphasizes flushing via forceFlush or sdk.shutdown in short-lived processes and environment-based credentials handling.

How it works

• Install core tracing: @langfuse/tracing, @langfuse/otel, and @opentelemetry/sdk-node.
• Use startActiveObservation() to create an observation, propagate context automatically, and end it when the callback completes.
• Use observe() to wrap functions for automatic input/output capture and timing.
• For OpenAI calls, wrap the client with observeOpenAI() to enable zero-config tracing.
• Manage prompts with LangfuseClient from @langfuse/client, including prompt.get, compile, and chat prompt handling.
• Attach scores/evaluations via LangfuseClient/langfuse.score.create with data types NUMERIC, CATEGORICAL, BOOLEAN, and optionally per-observation.
• Use datasets and experiments for testing; link prompts to traces as part of traces.
• Ensure flushing and shutdown: call forceFlush() or sdk.shutdown() in short-lived processes; call langfuse.score.flush() when needed.

When to use it

• Adding observability and tracing to LLM application code (any provider)
• Wrapping OpenAI SDK calls for automatic token/cost tracking
• Managing prompt templates with versioning, labels, and variable compilation
• Evaluating LLM output quality with scores (numeric, categorical, boolean)
• Running experiments against datasets for regression testing
• Tracking sessions, users, and metadata across multi-turn conversations
• Monitoring LLM costs and token usage in production

What it can touch

• OpenTelemetry setup via instrumentation.ts (top of entry point)
• Tracing APIs: startActiveObservation, observe, startObservation, and observeOpenAI
• Langfuse client: LangfuseClient, prompt.get, prompt.compile, score.create, score.flush
• OpenAI integration: @langfuse/openai wrapper
• Environment variables: LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, LANGFUSE_BASE_URL

Caveats

• Requires environment-based credentials (never hardcode credentials)
• OpenAI auto-instrumentation relies on observeOpenAI() for zero-config tracing; manual generation observations are discouraged
• Short-lived processes must flush/shutdown to avoid lost events
• Context propagation relies on using startActiveObservation() or observe() for nested tracing; manual startObservation() needs explicit .end() calls