GPT-3 is a large language model developed by OpenAI, trained on a vast amount of text scraped from the web, books, and other sources to generate text that closely resembles human writing. Released in 2020 with 175 billion parameters, it was, at the time, one of the largest language models ever trained, and it demonstrated that scaling up model size and training data could produce dramatic improvements in fluency and few-shot task performance without task-specific architecture changes.
GPT-3 powered the original public version of ChatGPT and a wide range of early third-party AI writing and coding tools, cementing the transformer architecture as the dominant approach in large language models. It has since been succeeded by newer generations, including GPT-3.5 and GPT-4, which built on the same core architecture with additional training, alignment techniques, and, in GPT-4's case, multimodal capabilities GPT-3 didn't have.
