Large Language Models
131 views
BPE
Quick Definition
Subword tokenization algorithm merging frequent character pairs
Full Definition
Byte-Pair Encoding, a subword tokenization algorithm iteratively merging frequent character pairs.
Examples
GPT tokenization, vocabulary building, subword splitting
Related Terms
tokenization
vocabulary
More Large Language Models Terms
Synthetic Data Generation
Using AI to create artificial training data replicating patterns
Masked Language Model
Training objective predicting randomly masked input tokens
Grounding LLM
Connecting LLM outputs to verifiable external sources
Adapter Layer
Small trainable modules in frozen transformer layers
Multi-Head Attention
Parallel attention operations concatenated for richer representations
Next Token Prediction
Core LLM objective predicting the next token from context