Large Language Models
22 views
Linear Attention
Quick Definition
Attention with linear complexity for long sequences
Full Definition
Attention mechanism with linear complexity for efficient long-sequence processing.
Examples
efficient transformers, long documents, streaming models
Related Terms
attention-mechanism
sparse-attention
More Large Language Models Terms
WordPiece
Subword tokenization splitting words into frequent units
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
Adapter Layer
Small trainable modules in frozen transformer layers
Mistral
Efficient open-source LLMs with strong performance
Attention Head
Individual attention mechanism learning specific patterns
KV Cache
Memory optimization storing key-values for faster generation