Large Language Models
283 views
Quantization LLM
Quick Definition
Reducing LLM weight precision for efficient inference
Full Definition
Reducing LLM weight precision to lower bits like INT8 or INT4 for efficient inference.
Examples
4-bit inference, GPTQ, AWQ, model deployment
Related Terms
quantization
inference-optimization
More Large Language Models Terms
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
Tool Use
Capability of LLMs to call external tools and APIs
Vector Database
Specialized database for high-dimensional embedding queries
Mixture of Experts
Architecture routing tokens to specialized expert sub-networks
Sparse Attention
Attention attending to only a subset of tokens
Sliding Window Attention
Attention limiting tokens to nearby window only