Large Language Models
266 views
Pre-training
Quick Definition
Initial training learning general language patterns from text
Full Definition
Initial training phase where models learn general language patterns from massive text corpora.
Examples
GPT pre-training, BERT training, foundation model creation
Related Terms
self-supervised-learning
fine-tuning
More Large Language Models Terms
Mistral
Efficient open-source LLMs with strong performance
Guardrails LLM
Frameworks for monitoring and controlling LLM I/O
Semantic Search
Search technique understanding meaning rather than keywords
Model Merging
Combining fine-tuned LLMs without additional training
Attention Head
Individual attention mechanism learning specific patterns
Prompt Caching
Storing key-value states for repeated prompt prefixes