Agent observability involves tracking every model call, tool execution, and reasoning step in a structured, replayable trace to detect failures that may appear as successful outcomes, such as accurate but incorrect outputs, unnecessary tool calls, or a plan deviating off course unnoticed.
Agent observability provides comprehensive insight into the actions taken by an AI agent in a production environment, including all model calls, tool selections, decision chains, memory retrievals, and agent-to-agent handoffs, all captured in a structured, hierarchical format that can be replayed. This surpasses traditional monitoring methods, which only confirm the service's availability without verifying if the agent made correct choices, passed accurate arguments, or followed its designated plan. The emerging standard that integrates these features is. OpenTelemetry's GenAI semantic conventions : a vendor-neutral vocabulary of gen_ai.* span and metric attributes that can be emitted by any instrumentation library and ingested by any backend, allowing teams to instrument once and easily switch observability platforms without having to re-instrument each trace point.
Conventional APM monitors the process of a request passing through middleware, interacting with a database, and generating a response with precision in milliseconds. An agent interprets a command, iterates through loops, utilizes various tools, assesses outcomes, and potentially utilizes additional tools before generating an outcome. Each iteration may lead to a different sequence of tools being activated, with task durations varying from 500 milliseconds to several minutes for complex processes.
Agents often fail in ways that appear successful, such as providing a polished but incorrect response, executing multiple tool commands unnecessarily to achieve the correct outcome, or performing an action that is technically correct but logically flawed. A dashboard indicating that the agent service is operational does not provide any insight into these potential failures.
One dialogue exchange can create seven or more layers of nested spans. When this is multiplied by five to ten exchanges, it becomes evident why production agents produce ten to a hundred times more telemetry than a standard API.
A basic agent trace schema must capture every potential issue in order to effectively address and fix failures. If certain types of problems are overlooked, such as tool-execution spans, then errors may go undetected, leading to actions being taken based on outdated data without any visible signs of a problem occurring. In such cases, the blame cannot be placed on the tool or the agent, but rather on the outdated information being used.
Model name, number of tokens, latency, and completion status for a single model call.
Name of the tool, its arguments, the raw output it generates, the duration of the tool execution, the
What was obtained, where it came from, and if it was pertinent to the task at hand.
The plan, the decision taken, and the transfer of control among agents or stages.
Before the adoption of a common standard, switching between observability backends required manually re-instrumenting every trace point. OpenTelemetry's GenAI semantic conventions have eliminated this tedious process by providing a standardized set of conventions. gen_ai.* Both instrumentation libraries can emit span and metric attributes that any OTLP-compatible backend can ingest.
After the spec was released, adoption of major agent frameworks quickly followed suit, with many now generating OTel-compliant spans either natively or through a lightweight instrumentation package. Leading backends are now able to read the standard directly. This means that developers only need to instrument once and can easily switch exporters instead of constantly rewriting instrumentation whenever the observability stack evolves.
Typically, instrumentation primarily captures metadata such as model name, token counts, latency, and finish reason, while intentionally excluding prompt content and tool arguments to avoid sensitive data exposure. Enabling content capture includes full prompt messages, system instructions, tool schemas, and results, which are beneficial for debugging purposes but require cautious handling instead of being enabled by default.
The majority of production setups are inherently designed to be multi-agent, with a planner assigning tasks to specialized agents that in turn utilize tools and services. The observability must demonstrate the complete journey of a single user request through the entire chain, rather than just focusing on the actions of individual agents in isolation.
The method involves context propagation, where trace context is carried with each agent or tool interaction. This allows for a clear visual hierarchy in the trace view: the root agent span is displayed at the top, with child spans for routing, planning, specialist agents, and tool usage nested below. This detailed view highlights the responsible agent at each stage, even when crossing queues and async workers.
Agent expenses can frequently be obscured within sub-agents, retries, and external API calls, rather than being directly attributed to the main request. This is why it is crucial for cost attribution to be granular, reaching down to every individual run, prompt, sub-agent, and even customer, instead of just focusing on the top-level agent. A documented real-world scenario resulted in costs exceeding thirteen thousand dollars in a single month. However, after implementing prompt caching, model routing, and redesigning the planning and execution process, the expenses were reduced by over ninety percent.
Pricing models can vary in complexity: certain platforms may charge only for LLM spans, while tool, retrieval, and embedding spans are included at no extra cost. This can result in a significantly lower cost for a system with numerous tool calls but few model calls compared to a flat per-span pricing model.
The OTel standard has led to the commoditization of tracing, with platform differentiation now focused on surfacing failed runs and converting production traces for the next test cycle.
Organizing options based on their operation style helps simplify decision-making: self-hosted solutions such as Grafana Tempo or open-source Phoenix are ideal for teams seeking complete data control; managed services like Datadog or Honeycomb offer less operational burden in exchange for some control; agent platforms like Braintrust or LangSmith seamlessly integrate tracing into the evaluation and release process.
| Platform | Best for | Model |
|---|---|---|
| Datadog / Honeycomb | Managed, AI-aware APM with native GenAI span support | Managed SDK |
| Arize Phoenix | Open-source, OTel-first tracing plus custom evaluators | Self-hosted |
| Langfuse | Open-source trace store with prompt management built in | Self-hosted |
| LangSmith | Native fit for LangChain / LangGraph agent stacks | Managed SDK |
| Braintrust | Trace-to-eval workflow, production failures become test cases | Trace-to-eval platform |
| Sentry | Agent traces unified with errors, replays, and infra traces | Managed SDK |
Teams that successfully implement tracing often share a common experience: when they can quickly diagnose production failures in minutes using traces instead of spending hours searching through unstructured logs, the benefits of early investment in tracing become clear and tangible.
Consider each user request as a standalone trace containing spans for planning, model calls, tool calls, and downstream services starting from the initial deployment, rather than treating it as an instrumentation task that is only addressed after a major outage occurs.
LLM monitoring tracks specific model call metrics like latency, tokens, and errors, while agent observability monitors the entire agent process including multi-step reasoning, tool execution, and agent-to-agent handoffs, providing insight into why an agent may have made more model calls than necessary.
The GenAI semantic conventions of OpenTelemetry establish a universal set of gen_ai.* attributes for LLM calls, tool invocations, and agent operations. Adhering to this standard allows an agent's telemetry data to seamlessly integrate with any OTLP-compatible backend, simplifying the process of switching observability platforms to just swapping out an exporter instead of redoing all instrumentation in the codebase.
LLM calls cover a range of information including the model, tokens, latency, and finish reason. Tool executions involve details such as the tool name, arguments, output, duration, and retries. Memory operations detail what was retrieved and from where. Agent orchestrations track the plan, decision-making process, and control passing between steps or agents. Failure to capture any of these elements can result in customer complaints.
By propagating context, the trace carries the root agent span at the top followed by nested child spans for routing, planning, specialist agents, and tool usage, even across queues and async worker processes, when one agent calls another agent or a tool.
Limiting the number of retries at the instrumentation level prevents a single failure cascade from creating an excessive number of spans. Tail sampling is used to retain all error traces while only capturing a portion of successful ones. Understanding the pricing model of their platform is important, as some only bill for LLM spans, making tool-heavy agents more cost-effective to observe compared to a flat per-span price.
By default, standard OTel exporters will gladly send complete prompts and tool-argument content to an observability backend if content capture is turned on. Due to this, most instrumentations only capture metadata such as model, tokens, and latency by default; enabling full content capture should include explicit masking or redaction for any sensitive fields.
From day one, ensure the four core span types are aligned with OpenTelemetry GenAI conventions, and choose an observability platform based on deployment model rather than feature checklist alone.