Large Language Models
25 views
Next Token Prediction
Quick Definition
Core LLM objective predicting the next token from context
Full Definition
The core training objective of autoregressive LLMs predicting the most probable next token given previous context.
Examples
GPT text generation, autocomplete, language modeling
Related Terms
autoregressive-model
large-language-model
pre-training
More Large Language Models Terms
Adapter Layer
Small trainable modules in frozen transformer layers
Speculative Decoding
Using draft model candidates verified by large model
QLoRA
Combining 4-bit quantization with LoRA for efficiency
Masked Language Model
Training objective predicting randomly masked input tokens
Pre-training
Initial training learning general language patterns from text
Inference LLM
Using a trained LLM to generate outputs from inputs