Large Language Models
26 views
Quantization LLM
Quick Definition
Reducing LLM weight precision for efficient inference
Full Definition
Reducing LLM weight precision to lower bits like INT8 or INT4 for efficient inference.
Examples
4-bit inference, GPTQ, AWQ, model deployment
Related Terms
quantization
inference-optimization
More Large Language Models Terms
Vector Database
Specialized database for high-dimensional embedding queries
Instruction Tuning
Fine-tuning on instruction-response pairs for better following
Alignment Tax
Performance cost of aligning AI with human preferences
Streaming
Sending LLM tokens to clients as they are generated
Function Calling
Structured mechanism for LLMs to generate API function calls
Next Token Prediction
Core LLM objective predicting the next token from context