Large Language Models
282 views
Multi-Head Attention
Quick Definition
Parallel attention operations concatenated for richer representations
Full Definition
A transformer mechanism running multiple attention operations in parallel and concatenating their outputs.
Examples
transformer layers, contextual learning, multi-perspective analysis
Related Terms
attention-mechanism
self-attention
transformer
More Large Language Models Terms
Mixture of Experts
Architecture routing tokens to specialized expert sub-networks
PEFT
Fine-tuning methods updating only small parameter subsets
Synthetic Data Generation
Using AI to create artificial training data replicating patterns
Streaming
Sending LLM tokens to clients as they are generated
Quantization LLM
Reducing LLM weight precision for efficient inference
Chain-of-Thought Prompting
Technique encouraging step-by-step reasoning in LLMs