Large Language Models
30 views
Temperature
Quick Definition
Parameter controlling randomness in text generation
Full Definition
A parameter controlling randomness in LLM outputs by scaling logits before sampling.
Examples
creative writing, deterministic output, sampling strategy
Related Terms
top-k-sampling
nucleus-sampling
More Large Language Models Terms
Positional Encoding
Incorporating token position information into transformers
Masked Language Model
Training objective predicting randomly masked input tokens
BERT
Google's bidirectional transformer for context understanding
Flash Attention
Memory-efficient attention using GPU SRAM block computation
RAG
Enhancing LLMs by retrieving relevant external knowledge
Multi-Head Attention
Parallel attention operations concatenated for richer representations