Artificial Intelligence
167 views
Self-Attention
Quick Definition
Mechanism allowing sequence positions to attend to all others
Full Definition
A mechanism allowing each position in a sequence to attend to all positions in the same sequence.
Examples
transformer layers, contextual embeddings, parallel processing
Related Terms
transformer
attention-mechanism
More Artificial Intelligence Terms
Reinforcement Learning
Learning through trial and error with reward feedback
Convolutional Neural Network
Neural network for image and grid-like data processing
Variational Autoencoder
Generative model using latent space encoding and decoding
GAN
Two competing neural networks generating synthetic data
Diffusion Model
Generative model creating data by iteratively removing noise
Sentiment Analysis
NLP technique determining emotional tone in text