In classical software, the attack surface is well-understood: SQL injection, XSS, buffer overflows, authentication bypass. Inputs are structured, outputs are deterministic, and security boundaries are clear. Decades of tooling (SAST, DAST, WAFs) exist to defend these surfaces.

AI introduces entirely new attack vectors: prompts are natural language (no schema to validate against), training data can be poisoned, model weights encode behavior that can be manipulated, tool calls extend the blast radius, and context windows mix trusted and untrusted content. None of these exist in classical software.

LLM01: Prompt Injection — Crafted inputs manipulate LLM behavior

LLM02: Sensitive Information Disclosure — Secrets, PII, or confidential data leak through responses

LLM03: Supply Chain — Compromised dependencies, models, or datasets

LLM04: Data and Model Poisoning — Training/fine-tuning/RAG data manipulated

LLM05: Improper Output Handling — Unvalidated LLM outputs cause downstream exploits

LLM06: Excessive Agency — Agents with too much autonomy or permissions

LLM07: System Prompt Leakage — Hidden prompts, policies, and tool schemas extracted

LLM08: Vector and Embedding Weaknesses — RAG stores become attack surfaces

LLM09: Misinformation — Confident falsehoods causing real harm

LLM10: Unbounded Consumption — Runaway cost, latency, or capacity abuse

MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) extends the widely-adopted MITRE ATT&CK framework to AI-specific threats. As of October 2025, it catalogs 15 tactics, 66 techniques, 46 sub-techniques, 26 mitigations, and 33 real-world case studies. Over 150 organizations use it.

Evasion — Fooling models at inference time (adversarial examples)
Poisoning — Corrupting training data or model weights
Privacy — Extracting training data or model internals
Abuse — Misusing legitimate AI capabilities for harm

Added 14 new agentic AI techniques through collaboration with Zenity Labs, addressing autonomous agent security risks. ~70% of ATLAS mitigations map to existing security controls, enabling practical integration with current SOC workflows.

Prompt injection — Manipulating model behavior through crafted inputs (Ch 2)
Jailbreaking — Bypassing safety alignment to elicit prohibited outputs (Ch 3)
System prompt extraction — Leaking hidden instructions and tool schemas (OWASP LLM07)

Data poisoning — Corrupting training/fine-tuning data to alter behavior (Ch 4)
Supply chain — Malicious models on Hugging Face, pickle exploits (Ch 4)
Sleeper agents — Backdoors that persist through safety training (Anthropic, Jan 2024)

RAG poisoning — Injecting malicious documents into vector stores (Ch 7)
Tool poisoning — Hiding adversarial instructions in MCP tool descriptions (Ch 9)
Agent exploitation — Chaining innocent tools into harmful operations (Ch 8)
Model extraction — Stealing model weights or training data (Ch 10)

Training data memorization — Models can regurgitate PII, API keys, or proprietary data from training sets. Membership inference attacks can determine if specific data was used in training (AttenMIA achieves 0.996 AUC). System prompt leakage exposes internal policies and tool schemas.

Prompt injection corrupts outputs — Attackers can make models produce false information, bypass safety filters, or execute unauthorized actions. Data poisoning corrupts the model itself. RAG poisoning inserts false context that the model treats as ground truth.

Unbounded consumption (OWASP LLM10) — Adversaries can trigger runaway costs through expensive queries, denial-of-service via jamming attacks on RAG systems, or resource exhaustion through recursive agent loops. A single malicious prompt can cost thousands in API fees.

The dominant incident type in 2024–2025. Scammers use synthetic video and voice to impersonate public figures in investment scams, romance fraud, and wellness product schemes. Combines deepfakes with social media ad targeting and cash-transfer funnels at scale.

Clinejection (Feb 2026): A malicious GitHub issue exploited Cline’s AI triage bot (Claude-powered), resulting in an unauthorized npm publish affecting 5M+ users. PickleRAT: Malicious models on Hugging Face execute arbitrary code via pickle deserialization, attributed to APT41 subgroup.

DOGE/ChatGPT grants (2025): The Department of Government Efficiency relied on unvetted ChatGPT outputs to cancel $100M+ in National Endowment for the Humanities grants. The AI provided biased and inaccurate assessments of grant proposals.

The pipeline in the header shows the defense-in-depth approach:

1. Input Guardrails — Screen user inputs for injection, toxicity, PII (Ch 6)
2. Model Safety — Alignment training, system prompts, safety classifiers
3. Tool Sandboxing — Least-privilege, capability controls, WASM isolation (Ch 8)
4. Output Filtering — Validate outputs before they reach users (Ch 6)
5. Monitoring — Log everything, detect anomalies, continuous red teaming (Ch 11, 14)

No single defense is reliable against all attacks. Prompt injection has no complete solution — every guardrail can be bypassed with enough effort. The goal is to make attacks expensive, detectable, and limited in blast radius. Each layer catches what the previous one missed.

Ch 2: Prompt Injection — Direct, indirect, real-world CVEs
Ch 3: Jailbreaking — Crescendo, many-shot, encoded payloads
Ch 4: Data Poisoning — Sleeper agents, supply chain, PickleRAT
Ch 5: Adversarial ML — FGSM, PGD, evasion attacks

Ch 6: Guardrails — NeMo, LLM Guard, Lakera, Guardrails AI
Ch 7: Securing RAG — CPA-RAG, jamming, access control
Ch 8: Securing Agents — STAC, sandboxing, AMLA
Ch 9: Securing MCP — Tool poisoning, rug pulls, CVE-2025-54136

Ch 10: Privacy — MIA, differential privacy, GDPR
Ch 11: Red Teaming — Garak, PyRIT, PromptFoo, MITRE ATLAS
Ch 12: Governance — EU AI Act, NIST AI RMF, ISO 42001
Ch 13: Architecture — Zero-trust, API gateways, secrets
Ch 14: Production — Defense in depth, incident response