Large Language Models
29 views
KV Cache
Quick Definition
Memory optimization storing key-values for faster generation
Full Definition
A memory optimization storing computed key-value pairs to avoid redundant attention computation.
Examples
inference optimization, generation speedup, memory management
Related Terms
attention-mechanism
inference-optimization
More Large Language Models Terms
Next Token Prediction
Core LLM objective predicting the next token from context
BPE
Subword tokenization algorithm merging frequent character pairs
Context Window
Maximum tokens a language model can process at once
WordPiece
Subword tokenization splitting words into frequent units
Vector Database
Specialized database for high-dimensional embedding queries
Positional Encoding
Incorporating token position information into transformers