Joint probability is the likelihood that two or more events happen together, rather than the probability of each event considered on its own. For two events A and B, it's written as P(A and B), and it captures how the occurrence of one event relates to the occurrence of the other, whether they're independent, correlated, or one directly influences the other.
Language models rely on joint probability constantly, even if the term itself stays behind the scenes: predicting the next word in a sentence is really a question of the joint probability of that whole sequence of words occurring together, and the model's job during training is to learn what word sequences are jointly likely versus vanishingly rare, which is what lets it generate text that reads as coherent rather than as a random string of individually plausible words.
