Large Language Models
277 views
Masked Language Model
Quick Definition
Training objective predicting randomly masked input tokens
Full Definition
A training objective where masked tokens are predicted from surrounding context.
Examples
BERT pre-training, fill-in-the-blank, contextual understanding
Related Terms
bert
pre-training
self-supervised-learning
More Large Language Models Terms
WordPiece
Subword tokenization splitting words into frequent units
Prompt Caching
Storing key-value states for repeated prompt prefixes
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
Llama
Meta AI's open-source large language models
GGUF
File format for LLMs optimized for local inference
Model Merging
Combining fine-tuned LLMs without additional training