LoRA
Quick Definition
Parameter-efficient fine-tuning using low-rank matrix decomposition
Full Definition
Low-Rank Adaptation, a parameter-efficient fine-tuning method injecting low-rank matrices into transformer layers.
Examples
efficient fine-tuning, resource-constrained training
Related Terms
fine-tuning
qlora
adapter
More Large Language Models Terms
Constitutional AI
Training AI to follow explicit principles for consistency
BPE
Subword tokenization algorithm merging frequent character pairs
Temperature
Parameter controlling randomness in text generation
Hallucination LLM
LLMs generating plausible but incorrect information
Positional Encoding
Incorporating token position information into transformers
Reward Model
Model scoring LLM outputs based on human preferences