Large Language Models
142 views
Top-K Sampling
Quick Definition
Generation strategy considering top k most probable tokens
Full Definition
A generation strategy considering only the top k most probable tokens at each step.
Examples
text generation, creative writing, chatbot responses
Related Terms
nucleus-sampling
temperature
beam-search
More Large Language Models Terms
RoPE
Position encoding using rotation matrices for relative positions
Adapter Layer
Small trainable modules in frozen transformer layers
Attention Head
Individual attention mechanism learning specific patterns
Sliding Window Attention
Attention limiting tokens to nearby window only
Tokenizer Vocabulary
Complete set of tokens a language model can process
Distillation LLM
Training smaller LLMs to replicate larger model behavior