Large Language Models
150 views
Quantization LLM
Quick Definition
Reducing LLM weight precision for efficient inference
Full Definition
Reducing LLM weight precision to lower bits like INT8 or INT4 for efficient inference.
Examples
4-bit inference, GPTQ, AWQ, model deployment
Related Terms
quantization
inference-optimization
More Large Language Models Terms
Speculative Decoding
Using draft model candidates verified by large model
Hallucination LLM
LLMs generating plausible but incorrect information
GPT
OpenAI's family of autoregressive large language models
Adapter Layer
Small trainable modules in frozen transformer layers
Nucleus Sampling
Generation using cumulative probability threshold for token selection
Mixture of Experts
Architecture routing tokens to specialized expert sub-networks