Data poisoning

An attack that corrupts the data an artificial intelligence (AI) model learns from or looks up, so the model later makes wrong decisions that benefit the attacker.

For example, a fraud ring could file false disputes so a bank's fraud model learns that legitimate purchases look risky, or plant fake documents in a company's knowledge base so an AI assistant gives customers wrong answers.

An integrity attack on a machine learning (ML) system that inserts or alters samples in its training data, fine-tuning data, or retrieval sources. The US National Institute of Standards and Technology (NIST) adversarial machine learning taxonomy, NIST AI 100-2 E2025, groups poisoning by the attacker's goal:

• Availability poisoning, which degrades the whole model.
• Targeted poisoning, which changes outputs for a few chosen cases.
• Backdoor poisoning, which plants a hidden trigger the model reacts to.
• Model poisoning, which tampers with the model itself.

Techniques include label flipping, which gives training examples wrong labels, and clean-label poisoning, which uses correctly labeled samples crafted to skew learning. NIST notes that systems retrained on live operational data are especially exposed.

Retrieval-augmented generation (RAG) systems, which let a large language model (LLM) look up documents before answering, add another target. A paper at USENIX Security 2025 showed that five malicious texts per target question, in a knowledge base of millions, could steer answers about 90% of the time.

Data poisoning sits next to terms that are easy to mix up:

• Prompt injection manipulates a model when it reads input, while poisoning changes what it learned or can retrieve beforehand.
• Model drift is a gradual, unintended loss of accuracy as real-world data changes, while poisoning is deliberate.

For more, see LLM04:2025 Data and Model Poisoning from the Open Worldwide Application Security Project (OWASP).