Attention Mechanism

Quick Definition

Component allowing models to focus on relevant input parts

A neural network component that allows models to focus on relevant parts of the input when producing output.

self-attention, cross-attention, multi-head attention

transformer self-attention

Removing redundant neural network connections to reduce size

AI model ability to resist adversarial perturbations

Designing models that learn how to learn new tasks

Autonomous software taking goal-directed actions

Technology converting images of text into machine-readable text

Structured graph of entities and their relationships