A recurrent neural network (RNN) is a type of neural network architecture built specifically to handle sequential data, like text, speech, or time-series data, by maintaining a form of internal memory that carries information from earlier steps in the sequence forward to later ones. This lets an RNN process a sentence word by word while still keeping track of context from words it already saw.
RNNs were the standard choice for natural language processing tasks for years, but they struggle with long sequences, since information from early in a long input tends to fade by the time the network reaches the end of it, a limitation known as the vanishing gradient problem. Transformer-based architectures, which process an entire sequence in parallel using attention instead of RNNs' step-by-step memory, have since replaced RNNs as the dominant approach behind ChatGPT and essentially every other modern large language model.
