LoRA
Quick Definition
Parameter-efficient fine-tuning using low-rank matrix decomposition
Full Definition
Low-Rank Adaptation, a parameter-efficient fine-tuning method injecting low-rank matrices into transformer layers.
Examples
efficient fine-tuning, resource-constrained training
Related Terms
fine-tuning
qlora
adapter
More Large Language Models Terms
Reward Model
Model scoring LLM outputs based on human preferences
Tokenization
Breaking text into tokens for language model processing
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Speculative Decoding
Using draft model candidates verified by large model
Semantic Search
Search technique understanding meaning rather than keywords
Fine-Tuning
Further training pre-trained models on specific domain data