Prompt injection

What is prompt injection?

Prompt injection is an attack that gets an AI system to follow somebody else's instructions. Adversarial text arrives through the input or through content the system retrieves, and the model treats it as direction rather than as data, because it has no reliable way to tell the difference.

It sits at the top of the OWASP Top 10 for LLM Applications, and it has held that position since the list was first published. That is unusual for a security list, which normally reshuffles as defences mature. Prompt injection has stayed at number one because the defences have not matured in the way the ranking implies they should.

The reason is structural rather than a matter of engineering effort. A model receives everything as one sequence of tokens: the operator's system instructions, the user's request, and whatever content the system retrieved on the user's behalf. All three look the same to it. Following instructions found in that sequence is the model's job, and nothing in the sequence carries a marker saying which instructions are legitimate.

What is the difference between direct and indirect prompt injection?

Direct injection is a user typing adversarial instructions into their own session. Indirect injection is instructions hidden inside content the system retrieves, so a third party attacks somebody else's session. The second is the one that matters, and the two are routinely conflated.

The distinction is not academic, because the threat models differ completely. Confusing them leads organizations to defend against the version that is mostly a policy problem while leaving the version that is a genuine security problem open.

Indirect injection is the serious case for a specific reason: the victim has no idea an attack is in progress and the attacker never touches the system directly. Somebody influences a document, and later a colleague's agent reads it while doing ordinary work. The instructions execute with that colleague's permissions, against that colleague's data.

The surface is also wider than people expect. Instructions can be hidden in a shared document, a supplier's PDF, an inbound email, a support ticket, a web page an agent browses, the output of a tool it calls, or, in multimodal systems, inside an image or audio file. Anything the agent reads is a candidate, which is covered from the pipeline side at data ingestion.

Memory poisoning: indirect injection that persists

A variant worth naming separately. Where an agent keeps memory across sessions, an injection that succeeds once can write a false fact into that store, and every later session reads it as established truth. The attack outlives the session it arrived in, and nothing about the stored fact indicates it was injected. Anything entering durable memory deserves the scrutiny you would apply to code rather than to a search result. See context engineering.

How does an indirect prompt injection actually work?

In four steps: an attacker plants instructions in content the target's agent will read, the agent retrieves it during ordinary work, the model treats the planted text as direction, and the agent acts using its own permissions. Nothing in the chain looks anomalous.

The sequence below describes the mechanism at a level useful for threat modelling. It deliberately avoids working payloads, because the defensive value is in understanding the shape rather than in reproducing the technique.

1. The attacker plants instructions where the agent will read them

   Anywhere they can influence content the target's system retrieves: a document in a shared repository, a public web page, an inbound email, a submitted ticket, a field in a record. The text is usually positioned to be invisible or unremarkable to a human reader while remaining legible to the model.

2. The agent retrieves it during legitimate work

   No exploit is required at this stage, and this is what makes it hard to detect. Somebody asks the agent a reasonable question, the retrieval layer returns the most relevant material, and the planted content is genuinely relevant. Ranking cannot distinguish it.

3. The model reads it as instruction

   The retrieved text enters the same token sequence as the system prompt and the user's request. The model was built to follow instructions it finds there. From its perspective nothing unusual has happened, and no error is raised at any point.

4. The agent acts, with its own permissions

   This is where an agent differs from a chatbot. A chatbot produces text somebody reads. An agent calls tools, so the injected instruction can reach whatever the agent can reach: sending a message, changing a record, querying a system, or transmitting data outwards. The permissions used are the agent's, which is why they matter more than any filter.

Until recently this was widely treated as theoretical. That changed with the first confirmed zero-click exploit against a production assistant, documented in the security literature as EchoLeak, which required no action from the victim at all. Academic benchmarking has also put numbers on it: work on tool-integrated agents has reported successful indirect injection in a substantial minority of attempts against capable models, with published figures in the range of roughly a quarter to a half depending on the setup. Treat those as indicative of magnitude rather than as a measurement of your own system.

Why can prompt injection not be fixed?

Because the fix that solved SQL injection is structurally unavailable. Parameterized queries work by separating code from data at the protocol level. A model receives natural language in one token stream, so there is no protocol layer at which that separation could be made.

The comparison with SQL injection is made constantly and usually stops at the analogy. Following it through is what explains why one was solved and the other has not been.

• SQL had a boundary to enforce. A parameterized query sends the statement and the values along separate paths. The database knows which is which by construction, so a value can never be executed as a command. The boundary is in the protocol, not in a filter.
• An LLM has no such path. System instructions, user input, and retrieved content arrive as one sequence. No token carries a privilege level. The model cannot consult a boundary that does not exist.
• Natural language cannot be validated the way structured input can. OWASP makes this point directly: unlike SQL injection, prompt injection cannot be reliably mitigated through schema validation, because natural language is flexible and unstructured. There is no grammar to check against.
• Following instructions is the feature. The behaviour being exploited is the behaviour you are paying for. A model that ignored instructions found in its context would be useless for most of what agents do.

Two consequences follow, and they should shape how you plan. Neither retrieval augmentation nor fine-tuning closes it, which OWASP states explicitly, so architectural changes marketed as solutions generally are not. And the likelihood of a successful injection cannot be driven to zero, which means any strategy resting entirely on prevention is resting on something unachievable. That is not a counsel of despair. It is the reason the next section is ordered the way it is.

What actually reduces prompt injection risk?

Split every mitigation into two groups: those that reduce the likelihood of an injection succeeding, and those that reduce the impact when one does. Likelihood measures are probabilistic and never reach zero. Impact measures are deterministic and provable, which makes them the durable layer.

Almost all published guidance presents mitigations as a flat list. Sorting them by which of the two they address tells you where to spend first, and it follows directly from the previous section.

Measures that reduce likelihood

Worth having, and none of them closes the gap. Injection detection classifiers, input and output filtering, hardening the system instructions, and repeated adversarial testing. Each lowers the probability that a given attempt succeeds. None can make it zero, and an attacker gets as many attempts as they want, which is the asymmetry that makes this band insufficient on its own.

One measure explicitly does not belong here. OWASP is direct that the system prompt should never be treated as a secret or relied on as a security control, and that authorization and privilege separation belong in deterministic systems outside the model. Instructions telling a model to refuse malicious requests sit in the same channel an attacker writes to.

Measures that reduce impact

These hold whatever the model decides, because the model is not the thing enforcing them. This is where the real defence lives.

• Least-privilege tooling. An injected instruction can only reach what the agent can reach. Narrow the tool set to the task rather than granting the identity of whoever built it, and most of the impact disappears before any filter runs.
• Scoped, short-lived credentials. The agent acts with a credential. What that credential can do, and for how long, is the actual blast radius.
• Deterministic egress control. Restrict where an agent can send data, by rule rather than by judgement. Exfiltration requires an outbound path; removing paths is more reliable than inspecting traffic.
• Human approval on the irreversible. Decided by reversibility rather than perceived risk. One gate before an action that cannot be undone is worth more than several on reads.
• Isolation of untrusted content. Keep retrieved material in a separate context or a sandboxed step so it cannot expand what the agent is permitted to do.

What is the difference between prompt injection and jailbreaking?

Different attacker, different victim, different fix. Jailbreaking makes a model violate its own provider's policies, usually with the user as the attacker. Injection makes a system violate its operator's intent, usually with a third party attacking a user. They get conflated constantly.

The confusion is understandable, because both involve adversarial text and both look like the model misbehaving. The distinction matters because the remedies live in different places and with different parties.

Two practical consequences. A model that becomes harder to jailbreak is not thereby harder to inject, because injection does not require the model to break any rule: it only requires the model to follow the wrong instructions, which is normal behaviour applied to adversarial input. And a vendor citing improved jailbreak resistance has not answered a question about injection, so the two claims should be examined separately during evaluation.

Two more terms worth separating while we are here. Goal hijacking is an injection that redirects the system to perform the attacker's task. Prompt leaking is an injection that extracts the system instructions themselves, which matters mainly because of what teams put in them rather than because the instructions are inherently secret. Both are injection outcomes rather than separate attack classes.