Large Language Models
290 views
Adapter Layer
Quick Definition
Small trainable modules in frozen transformer layers
Full Definition
Small trainable modules inserted between frozen transformer layers for parameter-efficient fine-tuning.
Examples
efficient fine-tuning, multi-task learning, modular adaptation
Related Terms
peft
lora
fine-tuning
More Large Language Models Terms
Pre-training
Initial training learning general language patterns from text
DPO
Alignment method bypassing reward model for preference learning
Hallucination LLM
LLMs generating plausible but incorrect information
Streaming
Sending LLM tokens to clients as they are generated
KV Cache
Memory optimization storing key-values for faster generation
Sparse Attention
Attention attending to only a subset of tokens