Large Language Models
23 views
BPE
Quick Definition
Subword tokenization algorithm merging frequent character pairs
Full Definition
Byte-Pair Encoding, a subword tokenization algorithm iteratively merging frequent character pairs.
Examples
GPT tokenization, vocabulary building, subword splitting
Related Terms
tokenization
vocabulary
More Large Language Models Terms
Distillation LLM
Training smaller LLMs to replicate larger model behavior
RLHF
Aligning LLMs with human preferences using reinforcement learning
Beam Search
Search algorithm exploring multiple output sequences
PEFT
Fine-tuning methods updating only small parameter subsets
Context Window
Maximum tokens a language model can process at once
Multi-Head Attention
Parallel attention operations concatenated for richer representations