LangWatch

Currai

Currai provides deep observability for AI agents, capturing production conversations, model responses, retrieved context, and tool calls. It surfaces user intents, failures, policy violations, and improvement opportunities to help teams iterate faster. Designed for product and engineering teams, Currai delivers actionable insights across web, API, and SDK integrations, enabling continuous monitoring and optimization of chatbots, voice assistants, and other AI agents in real‑time.

Langfuse

Langfuse is an open-source AI engineering platform that provides a comprehensive suite of tools for evaluating and observing large language models (LLMs). It offers features such as LLM evaluations, observability, metrics, prompt management, and a playground for testing and experimentation. Langfuse also integrates with popular tools and platforms like OpenTelemetry, LangChain, OpenAI SDK, and LiteLLM.

Comet

Comet is an AI developer platform covering two connected needs: traditional ML experiment tracking and management, and LLM and agent observability through its Opik product. On the MLOps side, it lets data scientists track and compare training runs, version models and datasets, and monitor production models, with support for frameworks like PyTorch, TensorFlow, Hugging Face, and scikit-learn. Opik, Comet's LLM observability and evaluation platform, adds tracing across 60+ integrations, automatic error detection with an AI assistant called Ollie that recommends fixes, test suites with LLM-as-a-judge evaluation, and production monitoring dashboards, including cost tracking for coding agents like Claude Code. Opik's core feature set is available as a free, self-hostable open-source download in addition to Comet's hosted cloud plans.

Weights & Biases

Weights & Biases (W&B) is an AI developer platform for building, training, and monitoring machine learning models and LLM-based applications. Its core Models product tracks experiments, hyperparameters, and results so teams can compare training runs, while a model and dataset registry handles versioning and lineage across a pipeline. The platform extends into production with Weave, a tool for tracing, evaluating, and monitoring LLM applications, plus serverless fine-tuning and reinforcement learning for large language models. It can be deployed as SaaS, on dedicated cloud infrastructure, or fully self-hosted for compliance-sensitive teams.

EvalCore

EvalCore is an open-source developer tool that lets AI engineering teams know when a change to a prompt, model, or dependency makes their LLM-powered app worse. Instead of forcing you into a proprietary SDK or test harness, it wraps around anything that speaks HTTP or shell. You describe an evaluation as a YAML file plus a JSONL dataset, point it at your target, then stack scorers on top to define what "good" means. The core innovation is the cassette: on the first live run, EvalCore calls the real model and records every request and response into a local SQLite cache keyed by a hash of the canonical request. That cassette can be committed to your repository. In CI, EvalCore replays the recording entirely offline with zero network calls, zero API keys, and zero cost, producing deterministic verdicts. The `--baseline main` flag runs a comparison against main and exits nonzero only on regressions, making it a natural pre-merge gate. The tool is designed to be lightweight and language-agnostic. It ships as a single dependency-free binary available via `cargo install` or prebuilt binaries for macOS, Linux, and CI runners. It works with OpenAI-compatible APIs, vLLM, Ollama, REST endpoints, shell commands, and OTel/OpenInference traces. With Apache-2.0 licensing, no server, no signup, and no telemetry, EvalCore positions itself as a simple, auditable way to ship AI changes confidently.