Phoenix

Phoenix is an open-source observability and evaluation platform designed by Arize for AI engineers developing complex LLM applications and autonomous agents. The system provides deep tracing capabilities, enabling developers to monitor prompt flows, retrieval-augmented generation steps, external tool calls, and final outputs across every execution cycle. By visualizing execution trees and latency breakdowns, engineering teams can pinpoint hallucinations, broken retrieval links, and performance bottlenecks in production environments. Phoenix includes a built-in Prompt IDE that allows users to test prompt modifications against benchmark datasets before deploying changes. The platform also automates response scoring through custom evaluator models, helping teams maintain response quality and build curated fine-tuning datasets directly from execution logs. Integrations with popular orchestration frameworks streamline continuous evaluation within existing development pipelines. While Phoenix delivers extensive monitoring tools, setting up local tracing environments and configuring custom evaluation metrics requires technical expertise and familiarity with Python telemetry standards.