Large Language Models
151 views
Inference LLM
Quick Definition
Using a trained LLM to generate outputs from inputs
Full Definition
The process of using a trained LLM to generate outputs from new inputs.
Examples
text generation, chat responses, content creation
Related Terms
training
inference-optimization
More Large Language Models Terms
KV Cache
Memory optimization storing key-values for faster generation
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Fine-Tuning
Further training pre-trained models on specific domain data
PEFT
Fine-tuning methods updating only small parameter subsets
Pre-training
Initial training learning general language patterns from text
Alignment Tax
Performance cost of aligning AI with human preferences