Artificial Intelligence
285 views
Attention Mechanism
Quick Definition
Component allowing models to focus on relevant input parts
Full Definition
A neural network component that allows models to focus on relevant parts of the input when producing output.
Examples
self-attention, cross-attention, multi-head attention
Related Terms
transformer
self-attention
More Artificial Intelligence Terms
Self-Attention
Mechanism allowing sequence positions to attend to all others
State Space Model
Architecture using linear attention for efficient sequence modeling
AI Agent
Autonomous software taking goal-directed actions
AI Fairness
Ensuring AI decisions do not discriminate against groups
Reward Modeling
Training models to predict human preferences for RLHF
Sentiment Analysis
NLP technique determining emotional tone in text