Large Language Models
255 views
BPE
Quick Definition
Subword tokenization algorithm merging frequent character pairs
Full Definition
Byte-Pair Encoding, a subword tokenization algorithm iteratively merging frequent character pairs.
Examples
GPT tokenization, vocabulary building, subword splitting
Related Terms
tokenization
vocabulary
More Large Language Models Terms
Adapter Layer
Small trainable modules in frozen transformer layers
Llama
Meta AI's open-source large language models
Constitutional AI
Training AI to follow explicit principles for consistency
Temperature
Parameter controlling randomness in text generation
Linear Attention
Attention with linear complexity for long sequences
Tokenization
Breaking text into tokens for language model processing