Large Language Models
264 views
Positional Encoding
Quick Definition
Incorporating token position information into transformers
Full Definition
Techniques incorporating token position information into transformer models.
Examples
sinusoidal encoding, RoPE, ALiBi, learned embeddings
Related Terms
transformer
attention-mechanism
More Large Language Models Terms
Sparse Attention
Attention attending to only a subset of tokens
Embeddings LLM
Dense vectors capturing semantic meaning of text
GGUF
File format for LLMs optimized for local inference
Temperature
Parameter controlling randomness in text generation
DPO
Alignment method bypassing reward model for preference learning
Grounding LLM
Connecting LLM outputs to verifiable external sources