Best LLM observability tools in 2026 for production AI agents

What LLM observability has to do in production

An AI agent in production is a chain of model calls, tool calls and retrievals, and when an answer comes out wrong you need every step that produced it. LLM observability tools record those steps as traces and add cost per run, prompt and model version history, alerts on cost, latency and quality, and online evaluation, where scorers grade live traffic so you catch a quality drop before a customer does.

I build and run production AI agents for operational teams, so my bias is towards tools that make one run explainable to the person who acts on it, not only to the engineer who built it. No vendor paid to be here. I checked every price on the vendor's own page on 25 September 2026, and prices here move often, so check again before you budget. The acquisitions older lists miss get their own section after the tools.

1. Langfuse

Langfuse is the open-source default. It traces every model call and tool step, tracks tokens and cost, versions prompts, and runs evaluations with LLM-as-a-judge scorers, annotation queues and datasets, fed by its SDKs or OpenTelemetry. Alerts on cost, latency and scores arrived in June 2026. ClickHouse bought Langfuse in January 2026, and Langfuse says no licence changes are planned.

It wins when you must self-host for data residency, because the MIT-licensed core has no usage limits. On the cloud, Hobby is free for 50,000 units a month, Core is 29 dollars a month with 100,000 units, Pro is 199 dollars a month, and extra units cost 8 dollars per 100,000. Every trace, observation and score is a unit, so many-step agents use them fast. It is the wrong buy if nobody will run Postgres, ClickHouse, Redis and object storage, which self-hosting involves.

2. LangSmith

LangSmith is LangChain's tracing, evaluation and monitoring platform, and it traces any framework through its SDK or OpenTelemetry. It adds prompt versioning, a playground, online evaluators, annotation queues, and alerts on errors, latency, cost and feedback scores sent to Slack, PagerDuty or a webhook.

It wins for teams on LangChain or LangGraph. Developer is free for one seat and 5,000 base traces a month, and Plus is 39 dollars per seat per month with 10,000 included, after which usage is billed in LangSmith units that today work out at 5 dollars per 1,000 base traces kept for 14 days. Longer retention costs extra, online evaluators can switch it on automatically, and new cloud traces are capped at 180 days from 14 September 2026. It is the wrong buy if you need to self-host on a small budget, because self-hosting is Enterprise only.

3. Arize Phoenix and Arize AX

Phoenix is Arize's free tool built on OpenTelemetry, covering tracing, evaluations, prompt versions and experiments, and it runs on a laptop, in Docker or on Kubernetes. Arize AX is the managed platform above it, adding online evaluations, monitors with alerts to email, Slack or PagerDuty, and enterprise controls.

Phoenix wins for teams who want OpenTelemetry-native tracing they run themselves. Its Elastic License 2.0 lets you run it freely but not offer it to others as a hosted service, so it is source-available rather than strictly open source. AX Free covers 25,000 spans a month, AX Pro is 50 dollars a month for 50,000 spans, and Enterprise is a custom quote with a self-hosted option. Pro is the wrong tier for a busy agent, because every model call, tool call and retrieval is its own span.

4. Datadog Agent Observability

Datadog renamed LLM Observability to Agent Observability. It traces prompts, retrieval, tool calls and agent decisions with latency, tokens and cost, runs built-in and custom evaluators, sits beside your APM and infrastructure data, and ingests OpenTelemetry GenAI spans.

It wins for teams already on Datadog, where one bill and one set of monitors beat another vendor. Only LLM spans, meaning calls to a model provider, are billed, while tool and retrieval spans are free, which suits agents. The free tier covers 40,000 LLM spans a month, Pro starts at 160 dollars a month billed annually for 100,000 LLM spans, and extra usage is 3.50 dollars per 10,000 on an annual commitment. It is the wrong buy if you are not already a Datadog customer, because much of the value is correlation with the rest of the platform.

5. Braintrust

Braintrust puts evaluation first. You build datasets, compare prompts and models in experiments and grade output with LLM, code or human scorers, then run the same scorers online against production logs, with SQL-based alerts to Slack or a webhook. It raised an 80 million dollar Series B in February 2026.

It wins for product teams whose main question is whether a change made the agent better or worse. Starter is free with 1 GB of processed data and 10,000 scores, Pro is 249 dollars a month with 5 GB and 50,000 scores, and Enterprise adds hybrid or on-premises deployment. It is the wrong buy if you want open source or only need cheap tracing.

6. Galileo

Galileo focuses on evaluation and guardrails. It distils expensive LLM-as-a-judge evaluators into small Luna models that run with low latency, which makes live scoring cheaper and lets scores act as runtime guardrails. Cisco completed its purchase of Galileo in May 2026 to strengthen agent monitoring in Splunk Observability Cloud.

It wins for enterprises that need guardrails as well as monitoring, and for organisations standardised on Splunk. The standalone product is still sold, with 5,000 free traces a month, Pro at 100 dollars a month billed yearly for 50,000 traces, and Enterprise with VPC or on-premises options. It is the wrong buy for a small team that mainly needs tracing, and before a multi-year deal, ask how it fits Cisco's plans for Splunk.

7. Weights and Biases Weave

Weave is the LLM side of Weights and Biases, owned by CoreWeave since May 2025. It traces calls, runs evaluations with scorers and LLM judges, monitors production traffic and redacts personal data, with an Apache 2.0 SDK.

It wins for teams that already train or fine-tune models on Weights and Biases. Free includes 1 GB of ingestion a month, Pro starts at 60 dollars a month with 1.5 GB for companies under 50 employees, and extra ingestion is 10 cents per MB, with self-managed deployment on Enterprise. It is the wrong buy for an operations team with no model training, or for agents that log whole documents, because pricing by the megabyte punishes heavy inputs.

8. Pydantic Logfire

Logfire is the Pydantic team's observability platform, built on OpenTelemetry, with strong LLM features. One agent run becomes one trace covering model calls, tool calls, database queries and API requests, with panels showing messages, tokens, cost and latency. Alerts are scheduled SQL queries sent to Slack, Opsgenie or a webhook.

It wins for Python teams, especially on Pydantic AI, who want one tool for application and agent. Personal is free for 10 million records a month, Team is 49 dollars a month with five seats and 2 dollars per extra million records, and Growth is 249 dollars a month. The SDK is MIT licensed, the platform closed source, and self-hosting Enterprise only. It is the wrong buy if the people reviewing agent output are not engineers, because it is built around SQL.

9. OpenLIT and the OpenTelemetry GenAI conventions

This is the route for instrumenting once and keeping your options open. OpenTelemetry's semantic conventions for generative AI cover model calls, agents, tool calls, token usage and MCP, and most tools on this page ingest OpenTelemetry traces. OpenLIT, under Apache 2.0, instruments LLM providers, frameworks and vector databases automatically and adds a self-hosted interface with tracing, cost tracking, a prompt hub and evaluations. Grafana Cloud's AI Observability uses its SDK.

It wins for teams already on Grafana or an OpenTelemetry collector, and as insurance against consolidation, because switching backends becomes a configuration change rather than a rewrite. It is the wrong buy if nobody will own a self-hosted stack, since there is no hosted version, and the GenAI conventions are still in development, so pin the versions you depend on.

What changed in 2025 and 2026

ClickHouse bought Langfuse in January 2026. Mintlify bought Helicone in March 2026, and Helicone now runs in maintenance mode, with security updates, bug fixes and new models still shipping, and Mintlify says it will help customers move to another platform, which is why it is not on my list. ServiceNow bought Traceloop, maker of the OpenLLMetry library, in a deal reported in March 2026. Cisco completed its Galileo purchase in May 2026, and CoreWeave has owned Weights and Biases since May 2025. Humanloop's team joined Anthropic and its platform shut on 8 September 2025, after which any customer data not exported became inaccessible.

Products changed too. LangSmith now bills in its own usage units and cut maximum cloud retention for new traces from 400 to 180 days, Langfuse added alerts, Datadog renamed its product, and the OpenTelemetry GenAI conventions moved to their own repository, still short of stable. If nothing above fits, two smaller independents are alive and selling. Laminar is open source under Apache 2.0 with cloud plans from 30 dollars a month, and HoneyHive is free for 10,000 events a month before an enterprise quote.

What to actually log for an agent in production

Every tool above stores traces. The harder question is what goes into them, whichever logo you pick. Here is the record I want for every run, tied together by one run ID that also points at the order, invoice or ticket the run was about.

The input exactly as it arrived, kept or referenced by a hash so you can replay it, with the prompt version, model and configuration that handled it. Every tool call with its arguments, result, latency and any retry or error, because in my experience most agent failures start there. The sources behind the answer, down to the document, page or record, so anyone can check a figure against its origin.

The output as delivered, with the result of any validation rules. The cost in tokens and money per step, so you can price an outcome such as one processed invoice. And the final human decision, whether the person approved, edited or rejected the output and what they changed. It is the field teams skip and the one I value most, because the edits become labelled examples for your evaluation set, and the approval rate is the plainest measure of whether the agent saves anyone time.

Two cautions. Mask personal data before it reaches a vendor, and match retention to how long you may need to answer a complaint or an audit. A 14-day default will not cover a quarterly review, so if the tool keeps less, write the run record to your own database too.

Comparison at a glance

How to pick

For a small team or a first agent, start free with Langfuse on its Hobby tier or self-hosted, Phoenix on a laptop, or Logfire if you work in Python. Instrument with OpenTelemetry where you can, so the choice stays reversible.

For a mid-size team with an agent in production, decide by where you already work. On LangChain, LangSmith. On Datadog, Datadog. If evaluation is the main job, Braintrust. If data must stay in your own infrastructure, self-hosted Langfuse or Phoenix. Whatever you choose, alert on cost per run and on an evaluation score from the first day.

For an enterprise, the questions are deployment, governance and vendor stability. Langfuse, LangSmith, Arize AX, Braintrust and Galileo all sell enterprise plans with self-hosted or hybrid options. Given how many vendors changed hands this year, ask each what happens to your data and contract after an acquisition, and keep instrumentation on OpenTelemetry so you can move.