Large Language Models
161 views
Synthetic Data Generation
Quick Definition
Using AI to create artificial training data replicating patterns
Full Definition
Using AI models to create artificial training data that replicates real data patterns and distributions.
Examples
data augmentation, privacy-safe datasets, domain adaptation
Related Terms
synthetic-data
generative-ai
data-augmentation
More Large Language Models Terms
Sliding Window Attention
Attention limiting tokens to nearby window only
Temperature
Parameter controlling randomness in text generation
Masked Language Model
Training objective predicting randomly masked input tokens
Multi-Head Attention
Parallel attention operations concatenated for richer representations
RoPE
Position encoding using rotation matrices for relative positions
Scaling Laws
Relationships between model size, data, compute, and performance