Speculative Decoding
技能 已验证 活跃Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.
To enable users to significantly speed up LLM inference and reduce latency by leveraging advanced decoding techniques like speculative decoding, Medusa, and Lookahead decoding.
功能
- Accelerates LLM inference using speculative decoding
- Implements Medusa's multiple decoding heads for faster generation
- Utilizes Lookahead Decoding (Jacobi iteration) for parallel token generation
- Provides code examples for integration with Transformers and vLLM
- Details training methods and hyperparameter tuning for Medusa and Lookahead
使用场景
- Optimizing LLM inference speed (1.5-3.6x speedup)
- Reducing latency for real-time applications (chatbots, code generation)
- Deploying models efficiently on limited compute hardware
- Generating text faster without quality loss
非目标
- Model architecture design beyond adding decoding heads
- Training large language models from scratch
- Providing inference servers (focus is on decoding techniques)
- Handling tasks outside of LLM inference optimization
Practical Utility
- info:Edge casesThe SKILL.md discusses hyperparameter tuning and method selection, which touches on optimizing performance but does not explicitly list failure modes with recovery steps.
Execution
- info:Pinned dependenciesDependencies are listed, but not explicitly pinned with lockfiles in the SKILL.md, which could lead to issues if newer versions break compatibility.
安装
请先添加 Marketplace
/plugin marketplace add Orchestra-Research/AI-Research-SKILLs/plugin install AI-Research-SKILLs@ai-research-skills质量评分
已验证类似扩展
Speculative Decoding
98Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques. Use when optimizing inference speed (1.5-3.6× speedup), reducing latency for real-time applications, or deploying models with limited compute. Covers draft models, tree-based attention, Jacobi iteration, parallel token generation, and production deployment strategies.
Performance Analysis
100Comprehensive performance analysis, bottleneck detection, and optimization recommendations for Claude Flow swarms
Next Cache Components
100Next.js 16 缓存组件 - PPR、use cache 指令、cacheLife、cacheTag、updateTag
MongoDB Connection Optimizer
100为任何支持的驱动程序语言优化 MongoDB 客户端连接配置（池、超时、模式）。在处理/更新/审查实例化或配置 MongoDB 客户端（例如，调用 `connect()` 时）、配置连接池、对连接错误（ECONNREFUSED、超时、池耗尽）进行故障排除、优化与连接相关的性能问题时，请使用此技能。这包括构建具有 MongoDB 的无服务器函数、创建使用 MongoDB 的 API 端点、优化高流量 MongoDB 应用程序、创建长期运行任务和并发性，或调试与连接相关的失败等场景。
One On Ones
100Design and run effective 1:1 meetings that build trust, develop people, and surface problems early. Covers cadence setup, agenda ownership, conversation frameworks, question banks, and handling difficult topics. Use when: a new manager learning to run 1:1s, resetting unproductive 1:1s that became status updates, onboarding a new direct report, preparing for a difficult performance conversation, building trust with a new team, or coaching through career development discussions.
Sql Optimization
100Universal SQL performance optimization assistant for comprehensive query tuning, indexing strategies, and database performance analysis across all SQL databases (MySQL, PostgreSQL, SQL Server, Oracle). Provides execution plan analysis, pagination optimization, batch operations, and performance monitoring guidance.