A foundational model is a large-scale AI model trained on broad, diverse data, giving it general capabilities that can be adapted to a wide range of more specific downstream tasks, rather than being built for one narrow purpose from the start. GPT-3 and GPT-4, the models underlying ChatGPT, are widely cited examples of foundational models.
Because they require enormous amounts of training data and compute, foundational models are typically expensive and time-consuming to build from scratch, which is why the more common pattern is for a small number of organizations to train them and then make them available, directly or through fine-tuning, for others to adapt to their own specific applications rather than every company training its own model independently.
