Large Language Models
272 views
KV Cache
Quick Definition
Memory optimization storing key-values for faster generation
Full Definition
A memory optimization storing computed key-value pairs to avoid redundant attention computation.
Examples
inference optimization, generation speedup, memory management
Related Terms
attention-mechanism
inference-optimization
More Large Language Models Terms
GPT
OpenAI's family of autoregressive large language models
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Grounded Generation
Generating outputs faithful to provided source documents
Reward Model
Model scoring LLM outputs based on human preferences
Inference LLM
Using a trained LLM to generate outputs from inputs
Distillation LLM
Training smaller LLMs to replicate larger model behavior