Large Language Models
22 views
Training LLM
Quick Definition
Adjusting parameters on large datasets to learn language
Full Definition
Adjusting model parameters on large datasets to learn language patterns through optimization.
Examples
pre-training, fine-tuning, RLHF
Related Terms
pre-training
fine-tuning
gradient-descent
More Large Language Models Terms
Scaling Laws
Relationships between model size, data, compute, and performance
Top-K Sampling
Generation strategy considering top k most probable tokens
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Streaming
Sending LLM tokens to clients as they are generated
System Prompt
Special prompt defining AI assistant behavior and constraints
Sliding Window Attention
Attention limiting tokens to nearby window only