PHOENIX-ARIZE
Phoenix Arize OSS LLM observability OpenTelemetry.
Définition
Phoenix Arize key features : (1) Tracing : OpenTelemetry-based instrumentation via auto-instrumentation libraries (openinference-instrumentation-openai + openinference-instrumentation-langchain + openinference-instrumentation-llama-index + etc.), traces collected by Phoenix backend (local + SaaS), web UI shows hierarchical traces. (2) OpenInference standard : semantic conventions for LLM operations OpenTelemetry, standardize trace attributes for cross-tool compatibility (Phoenix + Langfuse + LangSmith + DataDog + custom tools all consume same trace format), Arize AI leadership donating OpenInference to CNCF/OpenTelemetry. (3) Evaluations : LLM-as-Judge built-in templates (correctness + relevance + toxicity + hallucination + custom criteria), Python evaluators, Pairwise evaluators. (4) Datasets : manage curated test datasets, import production traces -> dataset for evaluation. (5) Experiments : track experiment runs with metrics across datasets versions, comparison side-by-side. (6) Prompt Playground : iterate prompts side-by-side LLMs (OpenAI + Anthropic + Mistral + Cohere). (7) Embedding analysis : visualise embedding spaces (UMAP + t-SNE) detect drift + clusters, unique strength Phoenix vs other tools. (8) Self-hosted : Phoenix OSS Docker + Helm Kubernetes deploy. Customers : Tinder + Toast + Cresta + IBM + Notion AI ~10000+ deployments 2024.
Origine
Arize AI fondee 2020 a Mill Valley California par Jason Lopatecki (ex-Uber Eats) + Aparna Dhinakaran (ex-Tubi Engineering) ; Series A $19M 2020 + Series B $38M 2022 + Series C $70M 2024 (TCV lead) ; Phoenix OSS launched 2022 ; ~10000+ Phoenix deployments 2024.
Exemple en contexte
Notion AI uses Phoenix Arize for AI features observability : OpenInference instrumentation traces millions LLM calls/day across Notion AI features (writing assist + Q&A + summarization), embedding analysis Phoenix UMAP visualises user queries clusters identify product opportunities + drift detection model performance degradation.
Termes liés
- LangSmith — alternative LLM observability.