Prompt injection occurs when untrusted input attempts to divert an AI system from its authorized instructions. It can appear in something as ordinary as a web page an assistant has been asked to summarize: alongside the requested information, the text contains an order to abandon the task.
Reading that order should not mean obeying it. The problem arises when the system treats material it is supposed to analyze as if it had authority to direct its behavior. The NIST definition places the risk precisely in the combination of inputs with different levels of trust.
To understand the scope, it helps to separate two questions: Who can introduce content into the task, and what can the application do with it?
When a document tries to give orders
Imagine someone asks an assistant: “Summarize the main arguments on this page.” The page contains information about a topic and, between its paragraphs, an instruction unrelated to the request: respond only with “summary unavailable.”
There are three distinct elements. The person has requested a summary; the page supplies the material for it; the inserted order tries to replace the original task. If the assistant follows that order and omits the summary, the content it consulted has displaced the authorized instruction.
This is a harmless hypothetical example, not a test that has been run or a phrase that works against every system. It illustrates the trust boundary: the boundary between instructions the application should follow and content it receives for processing.
Language models process both instructions and documents expressed in natural language. The application needs to maintain the distinction between them, even when they use the same language or appear together in the context of a request. A page does not gain authority merely by claiming to contain a high-priority order.
Key idea: Allowing an assistant to read a document does not give the document authority to direct the assistant.
Nor is finding an imperative verb enough to conclude that an attack is underway. A novel, a manual, or a security article may quote instructions as part of its content. What matters is the context and whether someone is trying to turn that material into an instruction that overrides the legitimate task.
Direct and indirect injection: where the diversion enters
In direct injection, the attempt arrives through an input the attacker supplies directly to the system. It may try to make an application ignore the rules that define its task. Authorization still matters: legitimately asking for a different format or correcting a request is not, by itself, an injection.
In indirect injection, the attempt travels inside a resource the application consults: a web page, a document, or information retrieved to answer a question. Whoever controls that resource can try to influence the assistant without writing in the user's chat. This is the distinction reflected in NIST's definition of indirect prompt injection.
The difference has a practical consequence: a user can make a perfectly legitimate request and still lead the system to manipulated content. Assessing only what the user types in the chat therefore misses part of the problem.
Researchers have studied the feasibility of this route. In 2023, Greshake and coauthors documented demonstrations of injection through retrieved data in real-world and experimental applications. Their work establishes the mechanism in particular systems; it does not show that every current application is vulnerable or establish a universal success rate.
How prompt injection relates to other concepts
Prompt engineering is the practice of writing instructions to obtain a useful result. A detailed request is not an attack simply because it is persuasive or complex. Injection presents a different problem: it attempts to make content without the necessary authority alter the system's authorized behavior.
The term jailbreak usually refers to attempts to bypass safeguards. There is overlap: OWASP describes it as a form of injection aimed at that goal. But diverting a summary and getting a model to bypass a safety restriction do not describe exactly the same outcome. It is useful to explain what each case seeks to achieve without imposing absolute boundaries on the terminology.
Prompt injection should also be distinguished from data poisoning. Under the NIST definition, the latter involves adversarial manipulation of training data. An injection can act while a system is in use, as it processes information to respond, without changing its training. Tampering with a page an assistant consults does not necessarily mean the model itself has been changed.
From an altered response to a real-world action
The summary example shows a diverted response. Other scenarios can affect information or actions, but their scope depends on how the application is built. OWASP links impacts to the system's context and integrations.
An assistant that only returns text may produce a biased explanation, omit relevant information, or include content unrelated to the task. That can harm someone who relies on the response, even without access to other tools.
If the application also consults private files or uses external services, other possibilities arise. To assess a potential data exposure, one must check what information was available and through what channel it could leave. To assess an action, one must examine which tools exist, what permissions they have, and which controls apply before execution.
Key idea: A manipulated output, an exposure of information, and an executed action are distinct outcomes. Evidence of one does not automatically prove the others.
A model does not acquire unlimited powers by producing a sentence. The application around it determines which requests can become actual operations. This distinction helps avoid both alarmism and excessive confidence: a system without tools still poses risks through its responses; one with broad permissions requires additional controls.
Defenses that work together
Prevention requires several measures. OWASP's recommendations include limiting capabilities, distinguishing external content, maintaining human oversight, and conducting adversarial testing. These measures reduce risk without guaranteeing absolute protection.
- Distinguish instructions from external content. The application should identify consulted material as data for the task. That separation helps, although it does not prove that the model will always respect it.
- Apply least privilege. The assistant should have access only to the information and operations needed for its function. Reducing its permissions limits the potential scope of a diversion.
- Check sensitive operations. Application restrictions should limit what can be executed. Human confirmation is useful when it clearly shows the proposed action and the information affected.
- Test adversarial scenarios. Evaluating manipulated documents and possible diversions can reveal failures under specific conditions. Passing some tests does not mean that every future case has been covered.
Adding a protective instruction or filtering certain expressions does not justify a promise of immunity. Nor does retrieving external information make everything retrieved trustworthy. The practical question is what happens if the model misinterprets that content, and which barriers still work in that case.
Privacy and responsibility when delegating tasks
Before connecting an assistant to private information or allowing it to act, consider which resources it needs, who can influence what it reads, and what consequences a diversion could have. This applies a digital threat model to a particular use of AI.
From the perspective of individual liberty and responsibility, preserving people's control over their data and actions taken on their behalf makes sense. This is an editorial design principle: grant proportionate powers and offer understandable choices when consequences affect the user or others.
A person using an assistant can review its results and limit the information they share. The application designer must anticipate that consulted resources may contain hostile instructions. Broad permissions granted by default do not become prudent just because a response sounds convincing.
To decide how much to delegate, the summary example offers a simple test of judgment: if the document succeeded in changing the task, would the result be incorrect text, an exposure of information, or an operation with real-world effects? The answer indicates where stricter controls are needed and which decisions should remain under human oversight.
Share