Large Language Models
152 views
Adapter Layer
Quick Definition
Small trainable modules in frozen transformer layers
Full Definition
Small trainable modules inserted between frozen transformer layers for parameter-efficient fine-tuning.
Examples
efficient fine-tuning, multi-task learning, modular adaptation
Related Terms
peft
lora
fine-tuning
More Large Language Models Terms
Model Merging
Combining fine-tuned LLMs without additional training
DPO
Alignment method bypassing reward model for preference learning
Distillation LLM
Training smaller LLMs to replicate larger model behavior
Masked Language Model
Training objective predicting randomly masked input tokens
Autoregressive Model
Generative model predicting next token from previous tokens
Temperature
Parameter controlling randomness in text generation