Large Language Models
143 views
BERT
Quick Definition
Google's bidirectional transformer for context understanding
Full Definition
Bidirectional Encoder Representations from Transformers, a Google model processing text in both directions.
Examples
search ranking, question answering, sentiment analysis
Related Terms
transformer
encoder-only
More Large Language Models Terms
Tool Use
Capability of LLMs to call external tools and APIs
BPE
Subword tokenization algorithm merging frequent character pairs
Synthetic Data Generation
Using AI to create artificial training data replicating patterns
Prompt Caching
Storing key-value states for repeated prompt prefixes
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Alignment Tax
Performance cost of aligning AI with human preferences