A corpus is a large, structured collection of text (or sometimes speech) used as the raw material for training or evaluating language models and other natural language processing systems. It can be general-purpose, drawing broadly from books, websites, and articles to teach a model everyday language, or domain-specific, built from legal documents, medical records, or scientific papers to specialize a model's knowledge in that field.
The size, diversity, and quality of a corpus directly shape what a trained model can do: a narrow or biased corpus tends to produce a model with correspondingly narrow or biased outputs, which is why building a large language model well involves careful curation, deduplication, and filtering of the training corpus, not just accumulating as much raw text as possible.
