PEFT Fine Tuning
技能 已验证 活跃Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
To enable users to fine-tune large language models efficiently on limited hardware by training only a small fraction of model parameters.
功能
- Parameter-efficient fine-tuning (PEFT) with LoRA, QLoRA, and 25+ methods
- Guidance for fine-tuning 7B-70B models on consumer GPUs
- Code examples for standard LoRA and memory-efficient QLoRA
- Detailed instructions on loading, merging, and managing adapters
- Performance benchmarks and troubleshooting for common issues
使用场景
- Fine-tuning large LLMs (7B-70B) with limited GPU memory
- Training less than 1% of model parameters for minimal accuracy loss
- Enabling multi-adapter serving from a single base model
- Rapid iteration on task-specific adapters for LLMs
非目标
- Full fine-tuning for maximum quality when compute budget is not a constraint
- Training models smaller than 1B parameters
- Scenarios requiring updating all model weights due to significant domain shift
工作流
- Install necessary libraries (PEFT, Transformers, PyTorch, etc.)
- Configure LoRA or QLoRA parameters (rank, alpha, target modules)
- Load base model and apply PEFT configuration
- Prepare dataset and tokenize data
- Train the model using provided training arguments
- Save and optionally merge the trained adapter
实践
- Fine-tuning methodology
- LLM optimization
- Adapter management
先决条件
- Python 3.8+
- pip install peft transformers torch bitsandbytes datasets accelerate
安装
npx skills add davila7/claude-code-templates通过 npx 运行 Vercel skills CLI(skills.sh)— 需要本地安装 Node.js,以及至少一个兼容 skills 的智能体(Claude Code、Cursor、Codex 等)。前提是仓库遵循 agentskills.io 格式。
质量评分
已验证类似扩展
Peft Fine Tuning
99Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
Fine Tuning Expert
98Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model.
Unsloth
98Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
Implementing Llms Litgpt
98Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
Implementing Llms Litgpt
100Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
Unsloth
100Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization