Artificial Intelligence
26 views
Self-Attention
Quick Definition
Mechanism allowing sequence positions to attend to all others
Full Definition
A mechanism allowing each position in a sequence to attend to all positions in the same sequence.
Examples
transformer layers, contextual embeddings, parallel processing
Related Terms
transformer
attention-mechanism
More Artificial Intelligence Terms
Synthetic Data
Artificially generated data mimicking real-world distributions
Quantization
Reducing model precision from 32-bit to lower bit representations
Natural Language Processing
AI field enabling computers to understand human language
Pose Estimation
Detecting human body positions and joint locations
Superintelligence
Hypothetical AI surpassing all human cognitive abilities
AI Chip
Specialized hardware for accelerating AI computation