Large Language Models
144 views
Pre-training
Quick Definition
Initial training learning general language patterns from text
Full Definition
Initial training phase where models learn general language patterns from massive text corpora.
Examples
GPT pre-training, BERT training, foundation model creation
Related Terms
self-supervised-learning
fine-tuning
More Large Language Models Terms
Tool Use
Capability of LLMs to call external tools and APIs
Scaling Laws
Relationships between model size, data, compute, and performance
Constitutional AI
Training AI to follow explicit principles for consistency
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions
Fine-Tuning
Further training pre-trained models on specific domain data
RAG
Enhancing LLMs by retrieving relevant external knowledge