Large Language Models
136 views
Positional Encoding
Quick Definition
Incorporating token position information into transformers
Full Definition
Techniques incorporating token position information into transformer models.
Examples
sinusoidal encoding, RoPE, ALiBi, learned embeddings
Related Terms
transformer
attention-mechanism
More Large Language Models Terms
BERT
Google's bidirectional transformer for context understanding
Large Language Model
Massive AI model trained on text for language understanding and generation
KV Cache
Memory optimization storing key-values for faster generation
Vector Database
Specialized database for high-dimensional embedding queries
Model Merging
Combining fine-tuned LLMs without additional training
Beam Search
Search algorithm exploring multiple output sequences