Ch 13 — Secure AI Architecture Patterns — Under the Hood

AI gateway, zero trust, defense-in-depth, secrets management, reference architectures
Index← High Level
Under the Hood
-
Click play or press Space to begin. Click any node for deep-dive details...
Step- / 10
AAI Gateway PatternCentralized proxy for all LLM traffic — auth, rate limiting, logging
1
dns
AI GatewaySingle entry point
for all LLM calls
speed
Rate LimitingPer-user, per-model
token budgets
2
key
Key ManagementRotate API keys
never in client code
3
arrow_downward Zero trust applied to LLM systems
BZero Trust for AI SystemsNever trust, always verify — applied to prompts, models, tools
verified_user
Zero TrustVerify every request
at every layer
4
lock
mTLS & AuthService-to-service
mutual TLS
shield
Input ValidationTreat all prompts
as untrusted
5
arrow_downward Defense-in-depth layering & secrets management
CDefense-in-Depth & Secrets ManagementLayered security controls, vault-based secrets, credential rotation
layers
Defense LayersInput → model →
output → tool
6
vault
Secrets VaultHashiCorp Vault
AWS Secrets Manager
autorenew
Key RotationAutomated rotation
short-lived tokens
7
arrow_downward Network isolation & model serving security
DNetwork Isolation & Model ServingVPC segmentation, private endpoints, model weight protection
lan
VPC IsolationPrivate subnets
no public internet
8
memory
Model ServingWeight encryption
TEE enclaves
9
arrow_downward Complete reference architecture
EReference ArchitectureEnd-to-end secure AI system design
monitoring
ObservabilityTraces, metrics
security events
10
architecture
Full ArchitectureReference design
all layers