Large Language Models
260 views
Next Token Prediction
Quick Definition
Core LLM objective predicting the next token from context
Full Definition
The core training objective of autoregressive LLMs predicting the most probable next token given previous context.
Examples
GPT text generation, autocomplete, language modeling
Related Terms
autoregressive-model
large-language-model
pre-training
More Large Language Models Terms
Tool Use
Capability of LLMs to call external tools and APIs
Tokenizer Vocabulary
Complete set of tokens a language model can process
Distillation LLM
Training smaller LLMs to replicate larger model behavior
Top-K Sampling
Generation strategy considering top k most probable tokens
LoRA
Parameter-efficient fine-tuning using low-rank matrix decomposition
Multi-Head Attention
Parallel attention operations concatenated for richer representations