Large Language Models
151 views
WordPiece
Quick Definition
Subword tokenization splitting words into frequent units
Full Definition
A subword tokenization algorithm that splits words into frequent subword units based on likelihood.
Examples
BERT tokenization, vocabulary building, unknown word handling
Related Terms
tokenization
bpe
sentencepiece
More Large Language Models Terms
Scaling Laws
Relationships between model size, data, compute, and performance
Inference LLM
Using a trained LLM to generate outputs from inputs
Autoregressive Model
Generative model predicting next token from previous tokens
Grounded Generation
Generating outputs faithful to provided source documents
Attention Head
Individual attention mechanism learning specific patterns
Llama
Meta AI's open-source large language models