Large Language Models
138 views
Linear Attention
Quick Definition
Attention with linear complexity for long sequences
Full Definition
Attention mechanism with linear complexity for efficient long-sequence processing.
Examples
efficient transformers, long documents, streaming models
Related Terms
attention-mechanism
sparse-attention
More Large Language Models Terms
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
KV Cache
Memory optimization storing key-values for faster generation
Embeddings LLM
Dense vectors capturing semantic meaning of text
Adapter Layer
Small trainable modules in frozen transformer layers
DPO
Alignment method bypassing reward model for preference learning
Instruction Tuning
Fine-tuning on instruction-response pairs for better following