For AI engineers, ML platform teams, and compliance officers needing visibility into LLM application performance, cost, quality, and safety in production. Covers tracing depth, evaluation capabilities, prompt management, integration breadth, and — for compliance buyers — audit logging and data residency.
Last checked October 15, 2026
Editorial independence: aicompliancevendors.com does not accept vendor payment for inclusion or ranking. Every pick below is editor-selected against the criteria stated on this page, and every factual claim is traceable to a cited public source.
Top picks: Langfuse — Teams prioritizing MIT-licensed self-hosted LLM engineering with maximum integration flexibility; Arize AI — Teams that want a free self-hosted tracer with a managed enterprise upgrade path; LangSmith — LangChain and LangGraph teams needing native framework observability. Plus 4 more vendors reviewed below. Last updated October 15, 2026; every entry cites public sources.
Evaluation framework: LLM-as-judge, heuristic, or human annotation capabilities.
Integration breadth: supports major LLM providers and agent frameworks.
Active development: features shipped in the 12 months preceding October 2026.
At least one publicly documented pricing tier.
Vendor product pages, documentation, and pricing pages were reviewed, and every pick was re-checked on October 15, 2026. Pricing verified against official pricing pages. We note who owns each product now: ClickHouse acquired Langfuse (January 2026), Dynatrace acquired Arize (completed October 1, 2026) and Cisco acquired Galileo (completed May 22, 2026), which is now sold as Splunk Agent Observability. WhyLabs is excluded (see below). Ranking favors feature completeness, pricing transparency, and production-scale readiness.
Langfuse is now owned by ClickHouse, which announced the acquisition on January 16, 2026; ClickHouse says the core stays MIT-licensed and Langfuse Cloud keeps running as a standalone service. Langfuse covers tracing, prompt management, evaluation (LLM-as-a-judge, heuristics and human review), experiments and annotation, is OpenTelemetry-native and lists 100+ integrations. Its repository has more than 30,000 GitHub stars, and Langfuse cites 90B+ observations a month. Annual SOC 2 Type II and ISO 27001 audits cover the cloud service. Cloud plans: Hobby free (50k units a month, 30 days of data access, two users), Core $29/month, Pro $199/month and Enterprise $2,499/month.
Strengths
MIT-licensed core is free to self-host, with no usage limits stated.
OpenTelemetry-native with 100+ integrations; no framework lock-in.
SOC 2 Type II and ISO 27001 audits; HIPAA-ready US cloud region.
Limitations
Some self-hosted features need a paid Enterprise Edition license key.
Cloud free tier limited to 50k units a month and two users.
Arize is now part of Dynatrace, which announced the $915 million deal on August 13, 2026 and completed it on October 1, 2026; Arize says it will keep supporting both Phoenix and AX, and its capabilities will be integrated into Dynatrace over time. Phoenix is released under the Elastic License 2.0 and is free to self-host; it is built on OpenTelemetry and traces LlamaIndex, LangChain and DSPy among other frameworks. Arize AX is the managed platform: Free (25k spans and 1 GB a month, 15-day retention), Pro at $50/month (50k spans, 10 GB, 30-day retention) and Enterprise at custom prices as SaaS or self-hosted. Arize's compliance page lists SOC 2 Type II, PCI DSS 4.0, HIPAA compliance and CSA STAR Level 1.
Strengths
Phoenix is free to self-host (Elastic License 2.0) and built on OpenTelemetry.
AX Pro published at $50/month.
SOC 2 Type II, HIPAA and PCI DSS 4.0, per Arize's docs.
Limitations
AX Free limited to 25k spans and 1 GB a month, with 15-day retention.
Arize will be folded into Dynatrace over time, with no published timeline.
LangSmith is built by LangChain Inc., the company behind LangChain and LangGraph. It offers native tracing for popular agent frameworks plus OpenTelemetry ingest and export, so non-LangChain stacks work too, and it automatically clusters traces to surface usage patterns, common agent behaviors and failure modes. Managed cloud is available on every plan; bring-your-own-cloud (hybrid) and self-hosted deployment, which let teams control where data lives, require Enterprise. Developer is free for one seat with 5,000 base traces a month; Plus is $39 per seat a month with 10,000 base traces; Enterprise is custom. Base traces are kept 14 days.
Strengths
Built by the team behind LangChain and LangGraph.
Automatic trace clustering for usage patterns and failure modes.
BYOC and self-hosted options for data residency (Enterprise plan).
Limitations
Maintained by LangChain Inc. — potential vendor alignment concern.
Developer free tier limited to one seat and 5,000 base traces a month, kept 14 days.
Braintrust's Loop is a built-in AI agent that investigates logs, experiments and datasets and, with your approval, creates and edits prompts, scorers and datasets. Traces live in Brainstore, Braintrust's own database, which it says is 80x faster on real-world benchmarks. Braintrust raised an $80M Series B led by ICONIQ in February 2026, with Andreessen Horowitz and Greylock returning, and names Notion, Replit, Cloudflare, Ramp and Dropbox as customers. Starter is free (1 GB of processed data and 10k scores a month, 14-day retention); Pro is $249/month (5 GB, 50k scores, 30-day retention); Enterprise adds BYOC and self-hosting.
Strengths
Loop agent investigates logs and experiments and edits prompts, scorers and datasets with your approval.
Brainstore trace database (vendor-reported 80x faster on benchmarks).
Unlimited users on every plan.
Limitations
BYOC and self-hosting require the Enterprise plan.
Pro at $249/month costs more than Arize AX Pro ($50/month), and more than LangSmith Plus ($39 per seat a month) for teams of up to six.
Fiddler is an AI Control Plane for agentic applications — observability, guardrails, and governance in one enterprise platform. Fiddler Guardrails, powered by its Centor models, cover PII, PHI and secrets, prompt injection and jailbreaks, hallucination and toxicity, with vendor-stated latency under 80 ms (checked October 2026). Plans: Free (includes guardrails), Developer at $0.002 per trace, and Enterprise on request.
Strengths
Built-in guardrails (Centor models) for PII, prompt injection, hallucination and toxicity.
Free plan includes guardrails; Developer plan priced per trace.
Every enforcement decision is logged, for auditable governance.
Limitations
Less open-source transparency than Langfuse or Arize Phoenix.
Galileo is now owned by Cisco, which announced the deal on April 9, 2026 and completed it on May 22, 2026; Galileo's documentation says that as of August 7, 2026 it is Splunk Agent Observability. The product evaluates, monitors and guards agents: evaluators score correctness, relevance, tool use and safety, and evaluations can become real-time guardrails for PII, PHI and PCI leakage, tool misuse and prompt injection. Splunk says its Luna models score 100% of production traffic at a lower cost than a frontier judge model. Splunk's pricing page lists Agent Observability from $100 a month per 1.2 million spans, billed annually, including 500 million Luna tokens a year, in Observability Cloud or self-hosted.
Strengths
Evaluations become real-time guardrails for PII, PHI and PCI leakage, tool misuse and prompt injection.
Luna models score all production traffic (per Splunk).
Available in Splunk Observability Cloud or self-hosted.
Limitations
Now sold through Splunk: galileo.ai/pricing redirects to Splunk, and Galileo's former Free and Pro plans are no longer published (checked October 2026).
Arthur monitors and evaluates models and agents. Its platform ingests spans and evals, tracks tokens and cost and supports OpenTelemetry, with listed integrations for OpenAI, the Anthropic SDK, AWS Bedrock, LangChain, LlamaIndex, CrewAI, the OpenAI Agents SDK and Google ADK. Its April 2026 release added an Agent Resource Graph that catalogs which models and tools each agent uses, plus tracing of Claude Code sessions as OpenInference spans. The Arthur Evals Engine is open source, with self-serve deployment. Plans: Free ($0, up to 4 use cases, 300k spans, 7-day retention), Premium ($60/month, up to 100 use cases, 1.2M spans, 30-day retention) and Enterprise (custom, with SSO, a BAA and VPC options).
Strengths
Free plan and a $60/month Premium plan are published.
Open-source Arthur Evals Engine with self-serve deployment.
Agent Resource Graph maps models and tools across agents (April 2026).
Limitations
Arthur does not say whether its span limits are monthly or total.
Criteria-based recommendations for the most common shortlist scenarios.
For free self-hosted observability, Langfuse (MIT-licensed core) is the default. For a free self-hosted tracer with a managed upgrade, Arize Phoenix and AX (now part of Dynatrace) fit. For LangChain-native teams, LangSmith is the tightest integration. For eval-driven development, Braintrust adds its Loop agent and Brainstore trace database. For regulated enterprises that need guardrails with an audit record, Fiddler AI fits; Splunk customers can use Galileo, now Splunk Agent Observability. For a low-cost start, Arthur publishes a free plan and a $60/month Premium plan.
What we did not include
Transparency about exclusions.
Patronus AI removed in October 2026: it now describes itself as a research lab developing training data and simulated environments for frontier AI, and its pricing page no longer lists prices. WhyLabs excluded: GeekWire reported on September 15, 2025 that Apple had acquired it, and WhyLabs says it is discontinuing operations and open-sourcing its platform. Arthur, excluded in April 2026 for lacking published LLM observability pricing, now publishes plans and was added in October 2026.
Frequently asked
What is the difference between LLM observability and LLM evaluation?+
LLM observability monitors production systems in real time: tracing requests, tracking latency, cost, error rates, and quality metrics over live traffic. LLM evaluation focuses on pre-deployment testing using datasets, metrics, and human annotation. Most platforms blend both.
Which LLM observability platform has the most generous free tier?+
Langfuse self-hosted (MIT-licensed core) states no usage limits. Langfuse Cloud Hobby: 50k units a month. Arthur Free: 300k spans (Arthur doesn't say whether per month or in total), 7-day retention. Arize AX Free: 25k spans and 1 GB a month. LangSmith Developer: 5k base traces a month. Braintrust Starter: 1 GB of processed data and 10k scores a month. Fiddler Free includes real-time guardrails. Galileo no longer publishes a free plan since becoming Splunk Agent Observability. Langfuse self-hosted or Cloud Hobby provides the highest-value entry point.