Large Language Models
22 views
PEFT
Quick Definition
Fine-tuning methods updating only small parameter subsets
Full Definition
Parameter-Efficient Fine-Tuning, a family of methods updating only a small subset of model parameters.
Examples
LoRA, prefix tuning, adapter layers, efficient adaptation
Related Terms
lora
fine-tuning
qlora
More Large Language Models Terms
BERT
Google's bidirectional transformer for context understanding
Speculative Decoding
Using draft model candidates verified by large model
Linear Attention
Attention with linear complexity for long sequences
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Nucleus Sampling
Generation using cumulative probability threshold for token selection
Tool Use
Capability of LLMs to call external tools and APIs