Large Language Models
33 views
Adapter Layer
Quick Definition
Small trainable modules in frozen transformer layers
Full Definition
Small trainable modules inserted between frozen transformer layers for parameter-efficient fine-tuning.
Examples
efficient fine-tuning, multi-task learning, modular adaptation
Related Terms
peft
lora
fine-tuning
More Large Language Models Terms
Sparse Attention
Attention attending to only a subset of tokens
Prompt Template
Reusable structured format with dynamic input placeholders
Sliding Window Attention
Attention limiting tokens to nearby window only
KV Cache
Memory optimization storing key-values for faster generation
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
DPO
Alignment method bypassing reward model for preference learning