Large Language Models
22 views
Positional Encoding
Quick Definition
Incorporating token position information into transformers
Full Definition
Techniques incorporating token position information into transformer models.
Examples
sinusoidal encoding, RoPE, ALiBi, learned embeddings
Related Terms
transformer
attention-mechanism
More Large Language Models Terms
Semantic Search
Search technique understanding meaning rather than keywords
Tokenizer Vocabulary
Complete set of tokens a language model can process
Sliding Window Attention
Attention limiting tokens to nearby window only
Grounded Generation
Generating outputs faithful to provided source documents
Adapter Layer
Small trainable modules in frozen transformer layers
Reward Model
Model scoring LLM outputs based on human preferences