LoRA
Quick Definition
Parameter-efficient fine-tuning using low-rank matrix decomposition
Full Definition
Low-Rank Adaptation, a parameter-efficient fine-tuning method injecting low-rank matrices into transformer layers.
Examples
efficient fine-tuning, resource-constrained training
Related Terms
fine-tuning
qlora
adapter
More Large Language Models Terms
Grounded Generation
Generating outputs faithful to provided source documents
Mistral
Efficient open-source LLMs with strong performance
Multi-Head Attention
Parallel attention operations concatenated for richer representations
Tokenization
Breaking text into tokens for language model processing
Llama
Meta AI's open-source large language models
KV Cache
Memory optimization storing key-values for faster generation