Large Language Models
187 views
Scaling Laws
Quick Definition
Relationships between model size, data, compute, and performance
Full Definition
Empirical relationships between model size, dataset size, compute, and performance.
Examples
Chinchilla, compute-optimal training, model sizing
Related Terms
large-language-model
training
More Large Language Models Terms
Context Window
Maximum tokens a language model can process at once
Hallucination LLM
LLMs generating plausible but incorrect information
Streaming
Sending LLM tokens to clients as they are generated
Llama
Meta AI's open-source large language models
Linear Attention
Attention with linear complexity for long sequences
Adapter Layer
Small trainable modules in frozen transformer layers