PEFT Fine Tuning
Skill Verifiziert AktivParameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
To enable users to fine-tune large language models efficiently on limited hardware by training only a small fraction of model parameters.
Funktionen
- Parameter-efficient fine-tuning (PEFT) with LoRA, QLoRA, and 25+ methods
- Guidance for fine-tuning 7B-70B models on consumer GPUs
- Code examples for standard LoRA and memory-efficient QLoRA
- Detailed instructions on loading, merging, and managing adapters
- Performance benchmarks and troubleshooting for common issues
Anwendungsfälle
- Fine-tuning large LLMs (7B-70B) with limited GPU memory
- Training less than 1% of model parameters for minimal accuracy loss
- Enabling multi-adapter serving from a single base model
- Rapid iteration on task-specific adapters for LLMs
Nicht-Ziele
- Full fine-tuning for maximum quality when compute budget is not a constraint
- Training models smaller than 1B parameters
- Scenarios requiring updating all model weights due to significant domain shift
Workflow
- Install necessary libraries (PEFT, Transformers, PyTorch, etc.)
- Configure LoRA or QLoRA parameters (rank, alpha, target modules)
- Load base model and apply PEFT configuration
- Prepare dataset and tokenize data
- Train the model using provided training arguments
- Save and optionally merge the trained adapter
Praktiken
- Fine-tuning methodology
- LLM optimization
- Adapter management
Voraussetzungen
- Python 3.8+
- pip install peft transformers torch bitsandbytes datasets accelerate
Installation
npx skills add davila7/claude-code-templatesFührt das Vercel skills CLI (skills.sh) via npx aus — benötigt Node.js lokal und mindestens einen installierten skills-kompatiblen Agent (Claude Code, Cursor, Codex, …). Setzt voraus, dass das Repo dem agentskills.io-Format folgt.
Qualitätspunktzahl
VerifiziertVertrauenssignale
Ähnliche Erweiterungen
Peft Fine Tuning
99Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.
Fine Tuning Expert
98Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model.
Unsloth
98Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
Implementing Llms Litgpt
98Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
Implementing Llms Litgpt
100Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
Unsloth
100Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization