Prompt injection is an attack technique where crafted text, embedded in a user message or in content the model reads, such as a webpage or document, tries to override a language model's original instructions and make it behave in an unintended way. It can be used to extract a hidden system prompt, bypass safety guidelines, or manipulate an AI-powered tool like a chatbot or coding assistant into producing output its developers didn't intend.
The risk grows as AI systems gain access to external tools and data sources: a model that reads emails, browses the web, or executes code on a user's behalf can potentially encounter injected instructions hidden in that external content, not just in the user's own message. Defending against prompt injection is an active area of security research, since there's no fully reliable way yet to guarantee a model will always distinguish trusted instructions from untrusted content it merely processes.
