Within a transformer model's architecture, attention is the mechanism that lets the model weigh how relevant every other word in a sequence is to understanding any given word, rather than processing each word in strict isolation or fixed order. This lets the model capture relationships between words that are far apart in a sentence just as effectively as relationships between words sitting right next to each other.
The technique was introduced in the landmark 2017 paper "Attention Is All You Need," which proposed building an entire model architecture, the transformer, around this mechanism instead of the recurrent, step-by-step processing earlier models relied on. That shift is what let models process full sequences in parallel and scale to the size of today's large language models, making attention arguably the single most consequential idea in the modern history of natural language processing.
