Inference is the act of using a trained model to generate predictions, classifications, or decisions on new data it hasn't seen before, as opposed to training, which is the process of teaching the model in the first place. Once a model has been trained and its parameters set, inference is what happens every time it's actually used: an image classifier inferring what's in a new photo, or a language model inferring the next word to generate given a prompt.
Inference and training have very different computational profiles: training is typically a one-time, extremely resource-intensive process performed by the model's developer, while inference happens repeatedly, often in real time, every time an end user interacts with the deployed model. This is why so much engineering effort goes into optimizing inference specifically, since a model that's cheap to train once but slow or expensive at inference time won't scale well to millions of everyday user requests.
