InstructGPT is a variant of GPT-3 that OpenAI trained specifically to follow explicit instructions in a prompt more reliably and to produce more helpful, honest responses than the base GPT-3 model did. Where the original GPT-3 was primarily trained to predict likely next words from web text, InstructGPT was further trained using human feedback to actually do what a user asked, rather than just producing plausible-sounding text.
This shift, using reinforcement learning from human feedback (RLHF) to fine-tune a base model toward following instructions, was a significant step in the development path that led directly to ChatGPT: the techniques introduced with InstructGPT became the template for how OpenAI, and much of the industry since, aligns large language models to be more directly useful and controllable rather than just fluent.
