What attackers do. This is a technique from MITRE ATLAS, the companion knowledge base for attacks on AI systems. Attackers craft input that causes a large language model to ignore its instructions or act on the attacker's instructions instead. The input can be typed directly or hidden in content the model reads, such as documents, web pages or emails, which is known as indirect prompt injection.
How to defend. Treat model output as untrusted, limit the tools and data an AI assistant can reach, and require human approval for sensitive actions. Separate trusted instructions from external content, filter inputs and outputs, and log prompts, retrieved content and tool calls for review. Test applications with adversarial prompts before release.
Signs to look for. Look for responses that reveal system instructions, unexpected tool calls or data access, and hidden instructions in retrieved content. A common gap is not logging what the model read, which makes indirect injection hard to trace.