Large Language Models
136 views
PEFT
Quick Definition
Fine-tuning methods updating only small parameter subsets
Full Definition
Parameter-Efficient Fine-Tuning, a family of methods updating only a small subset of model parameters.
Examples
LoRA, prefix tuning, adapter layers, efficient adaptation
Related Terms
lora
fine-tuning
qlora
More Large Language Models Terms
Sliding Window Attention
Attention limiting tokens to nearby window only
Adapter Layer
Small trainable modules in frozen transformer layers
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Tokenizer Vocabulary
Complete set of tokens a language model can process
Inference LLM
Using a trained LLM to generate outputs from inputs
Context Window
Maximum tokens a language model can process at once