Ch 12 — Security & Quality Risks

The vulnerabilities AI writes, the debt it creates, and how to defend against both
Index
High Level
shield
OWASP
arrow_forward
link_off
Supply
arrow_forward
code_off
Quality
arrow_forward
smart_toy
Agent
arrow_forward
gavel
Legal
arrow_forward
security
Defend
-
Click play or press Space to begin...
Step- / 8
shield
OWASP Top 10 in AI-Generated Code
The vulnerabilities AI writes most often — and why
The Scale of the Problem
~25% of AI-generated code contains confirmed OWASP vulnerabilities. AI optimizes for functionality, not security. Training data contains far more examples of insecure code than secure patterns. The model learns what’s common, not what’s safe.
The Top Vulnerabilities
A01: Broken Access Control Auth checks without authorization verification. Any authenticated user can access others’ data. Missing role-based checks. IDOR attacks. A03: Injection SQL injection via string concatenation. XSS through unescaped user input. NoSQL injection. SSRF (most common: 32 findings). A02: Cryptographic Failures Hardcoded secrets and API keys. Weak password hashing (MD5, SHA1). Missing HTTPS enforcement. Insecure randomness for tokens.
Why AI Writes Insecure Code
• Training data bias — tutorials and Stack Overflow answers prioritize clarity over security
• Happy-path optimization — AI generates code that works, not code that’s safe
• Missing threat model — AI doesn’t think about attackers unless prompted
• Context blindness — AI doesn’t know your deployment environment or compliance requirements
Model Variation
Security performance varies significantly by model. In 2026 benchmarks: GPT-5.2 had the lowest vulnerability rate at 19.1%, while several other models scored 29.2%. Choosing a more security-aware model matters, but no model is safe by default.
The uncomfortable truth: AI-generated code is now the cause of 1 in 5 security breaches. 69% of developers have discovered vulnerabilities introduced by AI code. This isn’t a future risk — it’s happening now.
link_off
Supply Chain & Hallucinated Dependencies
When AI recommends packages that don’t exist — and attackers fill the gap
Package Hallucination
AI models sometimes recommend packages that don’t exist. The model generates a plausible-sounding package name based on naming patterns in its training data. If you run npm install, it fails — unless an attacker has registered that name with malicious code. This is called dependency confusion or package squatting.
Outdated Dependencies
AI training data has a knowledge cutoff. The model may recommend library versions with known CVEs, deprecated APIs, or abandoned packages. It doesn’t check npm/PyPI for the latest secure version — it recommends what was popular in its training data.
Defense
1. Verify every package exists before installing 2. Check package age, maintainers, download count 3. Use lockfiles (package-lock.json, yarn.lock) 4. Run npm audit / pip audit after installs 5. Pin exact versions, not ranges 6. Use tools like Socket.dev or Snyk 7. Review AI-suggested imports manually
Key insight: Never blindly npm install a package the AI suggests. Verify it exists, check its reputation, and audit for known vulnerabilities. This 30-second check prevents supply chain attacks.
code_off
Code Quality Risks: The Subtle Problems
When code works but isn’t good
“Almost Right” Code
The most dangerous AI output: code that compiles, passes tests, and works in development but contains subtle logic errors. 66% of developers cite “almost right, but not quite” as their top frustration. This code is harder to debug than completely broken code because it creates false confidence.
Accidental Architecture
AI generates code that works but doesn’t fit your architecture. Different sessions produce different patterns. Over time, the codebase accumulates competing conventions: some files use one error handling pattern, others use another. This “accidental architecture” makes the codebase increasingly hard to maintain.
Technical Debt Acceleration
Vibe-coded projects accumulate technical debt 3x faster than traditionally developed software. Common debt patterns:

• Missing error handling on non-happy paths
• No input validation on internal functions
• Duplicated logic across files (AI doesn’t know about existing utils)
• Over-engineered solutions for simple problems
• Inconsistent naming and file organization
The 80/20 trap: AI code is often 80% correct. The remaining 20% contains the bugs, security holes, and architectural violations. But because 80% works, developers trust the whole thing. The discipline is in scrutinizing the 20% that matters most.
smart_toy
Agent-Specific Security Risks
When the AI tool itself becomes the attack surface
OWASP MCP Top 10 (2026)
OWASP released a dedicated Top 10 for AI agents using the Model Context Protocol:

Tool Poisoning (MCP-01): Attackers control tool descriptions to inject malicious instructions into agent reasoning
Excessive Permissions (MCP-02): Tools expose broad capabilities without least-privilege enforcement
Schema Misuse (MCP-03): Weak input schemas enable injection through AI drivers
Prompt Injection via Code
Malicious code in a repository can contain hidden instructions that manipulate the AI agent. Comments, strings, or file names can include text like “ignore previous instructions and...” that the agent may follow. This is especially dangerous when agents read untrusted code.
Data Exfiltration
An agent with network access could be tricked into sending sensitive code or credentials to an external server. Sandboxing and permission models (Ch 6) are the primary defense, but they must be configured correctly.
Defense: Treat AI agents like any other software with access to your codebase. Apply least-privilege permissions, sandbox execution, audit tool access, and never give agents unrestricted network access to production systems.
gavel
Legal & Licensing Risks
IP, copyright, and compliance in AI-generated code
Copyright Uncertainty
AI models trained on open-source code may generate output that closely resembles copyrighted code. The legal landscape is still evolving, but the risk is real: if AI generates code that’s substantially similar to GPL-licensed code, your proprietary project may have licensing obligations.
License Contamination
AI doesn’t track which training examples influenced its output. It may generate code patterns from GPL, AGPL, or other copyleft-licensed projects without attribution. For enterprise software, this creates compliance risk that legal teams are increasingly scrutinizing.
Mitigation Strategies
• Use AI tools with code attribution features that flag similar open-source code
• Run generated code through license scanning tools (FOSSA, Black Duck)
• Maintain a policy on AI-generated code in your organization
• Document which code was AI-generated for audit trails
• Review your AI tool’s terms of service regarding IP ownership
The evolving landscape: Courts are still deciding key cases about AI-generated code ownership. Until the law settles, treat AI output as requiring the same due diligence as code from any external source. When in doubt, rewrite.
key
Secrets & Credential Exposure
The #1 most preventable AI security failure
How It Happens
AI generates example code with hardcoded API keys, database URLs, and passwords. Developers copy the pattern without replacing the placeholder values. Or worse, developers paste real credentials into prompts, and the AI echoes them back into committed code. This is the single most common and most preventable AI security failure.
Defense Layers
1. .gitignore — exclude .env files 2. Pre-commit hooks — scan for secrets (gitleaks, detect-secrets, trufflehog) 3. CI pipeline — block commits with secrets 4. Rules file — “Never hardcode secrets” 5. Environment variables — always use .env 6. Vault/KMS — for production secrets
Never Paste Secrets into Prompts
Anything you type in a prompt may be logged, stored, or used for training. Never paste real API keys, passwords, database URLs, or tokens into an AI chat. Use placeholder values and configure real secrets through environment variables.
Non-negotiable: Install a pre-commit secret scanner today. It takes 5 minutes and prevents the most common AI security failure. No exceptions. No “I’ll do it later.” This is the single highest-ROI security investment for AI-assisted development.
security
The Security Review Checklist
What to check on every AI-generated PR
Authentication & Authorization
□ Are all endpoints authenticated? □ Is authorization checked (not just auth)? □ Are role checks on the server, not client? □ Can users access others’ data? (IDOR) □ Are tokens validated on every request?
Input & Output
□ Is user input validated and sanitized? □ Are SQL queries parameterized? □ Is output HTML-escaped? (XSS) □ Are file uploads restricted by type/size? □ Are redirects validated against allowlist?
Secrets & Configuration
□ No hardcoded secrets or API keys? □ .env files in .gitignore? □ HTTPS enforced? □ CORS configured restrictively? □ Error messages don’t leak internals?
Dependencies
□ All packages verified to exist? □ No known CVEs? (npm audit) □ Versions pinned, not floating? □ License compatible with your project?
Automate what you can: Pre-commit hooks for secrets, CI for dependency audits, linting for security patterns. Manual review for auth logic, business rules, and architectural decisions. The checklist ensures nothing is forgotten.
verified
Building a Security-First AI Workflow
Integrating security into every stage, not bolting it on at the end
Shift Left with Rules Files
Put security requirements in your rules file: “Always parameterize SQL queries. Always validate input with Zod. Never hardcode secrets. Always check authorization, not just authentication.” The agent follows these rules from the first line of code, not as an afterthought.
The Security-First Pipeline
1. Rules file → Security requirements upfront 2. Generation → AI follows security rules 3. Lint → Security-focused ESLint rules 4. Test → Security test suite (injection, auth) 5. Audit → npm audit, secret scan 6. AI review → Second model reviews for vulns 7. Human → Final review of auth & business logic 8. CI gate → All checks must pass to merge
The Numbers That Matter
Projects with structured AI workflows (rules files + security testing + review gates) produce 2.74x fewer security vulnerabilities than ad-hoc AI coding. The investment in security infrastructure pays for itself with the first prevented breach.
Key insight: Security in AI-assisted development isn’t about trusting or distrusting the AI. It’s about building a pipeline where vulnerabilities are caught regardless of their source — human or AI. The same defenses that catch AI mistakes catch human mistakes too.