Large Language Models
21 views
WordPiece
Quick Definition
Subword tokenization splitting words into frequent units
Full Definition
A subword tokenization algorithm that splits words into frequent subword units based on likelihood.
Examples
BERT tokenization, vocabulary building, unknown word handling
Related Terms
tokenization
bpe
sentencepiece
More Large Language Models Terms
Mixture of Experts
Architecture routing tokens to specialized expert sub-networks
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
GGUF
File format for LLMs optimized for local inference
Model Merging
Combining fine-tuned LLMs without additional training
DPO
Alignment method bypassing reward model for preference learning