Traditional perimeter security assumes everything inside the network is trusted. Zero Trust rejects implicit trust in any user, asset, or request. For AI systems, this is critical because:

Unpredictable outputs: LLMs generate dynamic requests that bypass traditional API validation
Machine speed: Agents operate without human oversight, making real-time verification essential
Multi-tenant risk: Shared model infrastructure means one compromised tenant can affect others
Tool chaining: Agents call external tools that expand the attack surface with every hop

1. Continuous verification: Authenticate every request, not just the connection. Per-request token validation, not session-based auth.

2. Least privilege: Grant minimum required permissions per tool call. Scope tokens to specific actions (Ch 8, Ch 9).

3. Micro-segmentation: Isolate model, application, integration, and infrastructure layers. A compromised guardrail shouldn’t expose the model weights.

4. Assume breach: Design for the scenario where any single component is compromised. Layered defenses ensure no single failure is catastrophic.

An LLM gateway sits between applications and LLM providers, enforcing security controls at a single chokepoint. It mirrors provider endpoints (e.g., OpenAI’s /v1/chat/completions), so applications redirect traffic by changing only the base URL. All requests and responses pass through the gateway’s security pipeline.

1. Authentication: Per-client API keys with 256-bit entropy
2. Rate limiting: Token-based (not request-based) with sliding windows
3. Model allowlist: Restrict which models clients can access
4. Prompt injection detection: 20+ regex patterns with cumulative risk scoring
5. PII scanning: SSN, credit cards, emails, phones — redact or block
6. Response scanning: Same injection/PII checks on LLM output
7. Provider routing: Load balance across OpenAI, Bedrock, etc.
8. Audit logging: Structured JSON with latency, correlation IDs

LLM API auth faces specific risks:

Bearer token exposure: Keys leak via git history, browser network requests, mobile binaries, and client-side JavaScript
Horizontal escalation: Accessing other users’ conversations
Vertical escalation: Regular users accessing admin functions
Parameter scope bypass: Clients overriding system prompts via API parameters

Replace static API keys with short-lived OAuth2 tokens, role-based scopes, and attribute-based access control.

Traditional request-based rate limiting is insufficient for LLMs — identical HTTP requests can vary dramatically in resource cost (a 10-token prompt vs. a 10,000-token prompt). Modern approaches limit tokens consumed, not requests sent.

Apache APISIX ai-rate-limiting plugin supports configurable token limits across sliding windows by token type (total, prompt, or completion tokens).

Enterprises need per-client limits, spike arrest, model failover, and fine-grained attribution — none of which provider defaults offer.

An LLM firewall is a specialized security layer that protects LLM applications from AI-specific threats. Unlike traditional WAFs, these address prompt injection, unsafe code generation, agent misalignment, and goal hijacking. They operate as real-time, production-ready layers supporting high-throughput environments.

Open-source guardrail framework with multiple specialized scanners:

PromptGuard 2: Lightweight classifier detecting direct prompt injection with high precision and low latency
AlignmentCheck: Chain-of-thought auditing that inspects agent reasoning for goal hijacking and indirect injection
CodeShield: Static analysis preventing insecure code generation across 8 programming languages
Custom scanners: Regex-based and LLM-prompt scanners for flexible threat detection

Akamai Firewall for AI: Enterprise LLM security for hybrid environments. Protects against prompt injection, data exfiltration, and model abuse at the edge.

Cloudflare AI Security for Apps: Integrated into their WAF. Prompt injection detection, unsafe/custom topic detection, PII detection and prevention. Available on all Cloudflare plans.

Traditional encryption protects data at rest (stored) and in transit (network). But during inference, data must be decrypted for the GPU to process it — creating a window where sensitive data is exposed in memory. Confidential computing closes this gap by encrypting data during processing using hardware-level Trusted Execution Environments (TEEs).

NVIDIA HGX B200 (Blackwell): Production-grade confidential computing with near-native performance. The historical 30–50% performance penalty has been eliminated.

NVIDIA H100: GPU TEE with 4–8% throughput penalty that diminishes with larger batch sizes.

CPU TEEs (Intel TDX/SGX): Under 10% throughput overhead and 20% latency overhead for full LLM inference pipelines.

SecureInfer partitions LLM workloads strategically:

• Security-sensitive components (attention projections, LoRA adapters) execute inside SGX enclaves
• Compute-intensive operations (matrix multiplication) run on GPUs after encryption

This balances security with performance for models like LLaMA-2. Source: arxiv.org/abs/2510.19979

AI-SPM is a cloud security category that addresses the rapid, often uncontrolled expansion of AI deployments. Three core capabilities:

1. Discovery & visibility: Find all AI applications, models, and infrastructure — including shadow AI that bypasses security review
2. Risk identification: Misconfiguration detection, vulnerability scanning, attack path analysis across AI pipelines
3. Data protection: Monitor sensitive data in training datasets, detect data leakage paths

Wiz: AI-BOM discovery, pipeline misconfiguration detection, attack path analysis extended to AI services

Orca Security: Agentless visibility covering 50+ AI models and packages, DSPM for AI training data

Palo Alto (Cortex Cloud): AI ecosystem visibility, model risk analysis, data classification across training pipelines

Microsoft Defender for Cloud: Multicloud AI workload discovery (Azure, AWS, GCP) with AI-BOM generation

Layer 1 — Edge (Gateway): LLM gateway with auth, rate limiting, model allowlists. Single security chokepoint for all AI traffic.

Layer 2 — Input (Firewall): LLM firewall (LlamaFirewall, Cloudflare) for prompt injection detection, PII scanning, input validation.

Layer 3 — Model (Isolation): Confidential computing (TEE-GPU), micro-segmentation, context window hygiene, memory isolation.

Layer 4 — Tools (Sandbox): WASM/container sandboxing (Ch 8), least privilege, human-in-the-loop for high-stakes actions.

Layer 5 — Output (Guardrails): Response scanning, output filtering (Ch 6), canary token detection.

Layer 6 — Observability (AI-SPM): Shadow AI discovery, anomaly detection, audit logging, compliance monitoring.

OWASP’s GenAI Security Solutions Reference Guide (2025) defines a structured LLMSecOps lifecycle across four phases:

Planning: Threat modeling (MITRE ATLAS), risk assessment (NIST AI RMF)
Data handling: PII scanning, data governance, AI-BOM
Deployment: Gateway, firewall, sandboxing, confidential computing
Monitoring: AI-SPM, red teaming (Ch 11), incident response