Large Language Models
21 views
Scaling Laws
Quick Definition
Relationships between model size, data, compute, and performance
Full Definition
Empirical relationships between model size, dataset size, compute, and performance.
Examples
Chinchilla, compute-optimal training, model sizing
Related Terms
large-language-model
training
More Large Language Models Terms
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Streaming
Sending LLM tokens to clients as they are generated
Context Window
Maximum tokens a language model can process at once
Quantization LLM
Reducing LLM weight precision for efficient inference
Instruction Tuning
Fine-tuning on instruction-response pairs for better following
WordPiece
Subword tokenization splitting words into frequent units