An autoregressive model predicts a value by looking at a sequence of its own prior values, using patterns from the past to forecast what comes next rather than treating each data point as independent. In statistics, this classically applies to things like predicting tomorrow's temperature from the previous several days' readings.
Large language models like ChatGPT are autoregressive in exactly this sense, but applied to language: the model predicts the next word in a sentence based on all the words that came before it, then feeds that predicted word back in as part of the context for predicting the one after it. This word-by-word, feed-forward process, repeated until the response is complete, is the core mechanism behind how these models actually generate text.
