Arize Prompt Optimization
Skill Verified ActiveOptimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement.
To enable users to systematically improve LLM prompt performance by analyzing production trace data and applying a data-driven optimization process.
Features
- Optimize LLM prompts using production trace data
- Extract prompts from spans
- Gather LLM performance signals
- Run data-driven optimization loops with `ax` CLI
- Debug and improve LLM output quality
Use Cases
- When needing to improve AI response quality
- For prompt engineering and tuning
- When improving system prompts based on performance metrics
- When analyzing LLM output for correctness and faithfulness
Non-Goals
- Directly modifying LLM models
- Collecting trace data (relies on external instrumentation)
- Managing Arize platform infrastructure
Workflow
- Extract the current prompt from production trace data
- Gather performance data from traces, datasets, and experiments
- Analyze failures and identify patterns for optimization
- Generate a revised prompt using a meta-prompt template
- Apply the revised prompt and iterate on the optimization loop
Prerequisites
- Requires the `ax` CLI
- Requires a configured Arize profile
Installation
npx skills add github/awesome-copilotRuns the Vercel skills CLI (skills.sh) via npx — needs Node.js locally and at least one installed skills-compatible agent (Claude Code, Cursor, Codex, …). Assumes the repo follows the agentskills.io format.
Quality Score
VerifiedSimilar Extensions
Prompt Optimization
100Applies prompt repetition to improve accuracy for non-reasoning LLMs
Arize Ai Provider Integration
100Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and other Arize features. Supports any LLM provider (e.g. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM). Use when the user mentions AI integration, LLM provider credentials, create integration, list integrations, update credentials, delete integration, or connecting an LLM provider to Arize.
Oh My Claudecode
100Process-first advisor routing for Claude, Codex, or Gemini via `omc ask`, with artifact capture and no raw CLI assembly
Unsloth
100Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
CE Optimize
100Run metric-driven iterative optimization loops -- define a measurable goal, run parallel experiments, measure each against hard gates or LLM-as-judge scores, keep improvements, and converge on the best solution. Use when optimizing clustering quality, search relevance, build performance, prompt quality, or any measurable outcome that benefits from systematic experimentation.
Arize Experiment
100Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance. Covers experiment CRUD, exporting runs, comparing results, and evaluation workflows using the ax CLI. Use when the user mentions create experiment, run experiment, compare models, model performance, evaluate AI, experiment results, benchmark, A/B test models, or measure accuracy.