Large Language Models
29 views
Speculative Decoding
Quick Definition
Using draft model candidates verified by large model
Full Definition
Using a small draft model to generate candidates verified in parallel by the large target model.
Examples
inference acceleration, LLM serving, throughput optimization
Related Terms
inference-optimization
kv-cache
More Large Language Models Terms
DPO
Alignment method bypassing reward model for preference learning
Attention Head
Individual attention mechanism learning specific patterns
Mixture of Experts
Architecture routing tokens to specialized expert sub-networks
Temperature
Parameter controlling randomness in text generation
Quantization LLM
Reducing LLM weight precision for efficient inference
BERT
Google's bidirectional transformer for context understanding