Phoenix
技能 已验证 活跃Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.
To provide a self-hosted, open-source solution for observing and evaluating LLM applications, enabling detailed tracing, systematic dataset evaluations, and real-time production monitoring.
功能
- LLM tracing with OpenTelemetry
- LLM evaluation framework
- Dataset management for regression testing
- Experiment pipelines for prompt/model comparison
- Self-hosted observability without vendor lock-in
使用场景
- Debugging LLM application issues with detailed traces
- Running systematic evaluations on datasets
- Monitoring production LLM systems in real-time
- Building experiment pipelines for prompt/model comparison
非目标
- Managed observability platform with vendor lock-in
- Deep learning experiment tracking focus
- General ML lifecycle management
- LangChain-first integration
工作流
- Launch Phoenix server (notebook or command-line)
- Configure OpenTelemetry tracing
- Instrument LLM framework (e.g., OpenAI, LangChain)
- Generate and collect traces
- Run evaluations on datasets
- Monitor and analyze LLM application performance
实践
- Observability
- LLM Tracing
- Evaluation
- Monitoring
先决条件
- Python 3.8+
- pip install arize-phoenix
Compliance
- info:GDPRThe tool collects telemetry about LLM application usage, which may include personal data if not properly anonymized by the user. No specific sanitization for GDPR is detailed.
- info:Telemetry opt-inThe extension collects telemetry for LLM application observability. While not explicitly stated as opt-in by default, the nature of collecting application-level data suggests it's part of the core functionality rather than optional tracking.
Execution
- info:Pinned dependenciesWhile dependencies are declared, explicit pinning via lockfiles or version constraints in SKILL.md are not clearly presented, though standard Python packaging should manage this.
安装
请先添加 Marketplace
/plugin marketplace add Orchestra-Research/AI-Research-SKILLs/plugin install AI-Research-SKILLs@ai-research-skills质量评分
已验证类似扩展
LangSmith Observability
99LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.
Observability Designer
100Observability Designer (POWERFUL)
Grafana Dashboards
99Create and manage production Grafana dashboards for real-time visualization of system and application metrics. Use when building monitoring dashboards, visualizing metrics, or creating operational observability interfaces.
Monitor Stream
99Stream live swarm events using the Monitor tool for real-time observability
Instrument Distributed Tracing
99Instrument applications with OpenTelemetry for distributed tracing, including auto and manual instrumentation, context propagation, sampling strategies, and integration with Jaeger or Tempo. Use when debugging latency issues in distributed systems, understanding request flow across microservices, correlating traces with logs and metrics for root cause analysis, measuring end-to-end latency, or migrating from legacy tracing systems to OpenTelemetry.
Sentry Feature Setup
99Configure specific Sentry features beyond basic SDK setup. Use when asked to monitor AI/LLM calls, set up OpenTelemetry pipelines, or create alerts and notifications.