Ch 7: Context Engineering

lightbulb

What Is Context Engineering?

The skill that separates frustrating AI sessions from productive ones

The Definition

Context engineering is the art and science of filling the context window with exactly the right information for the next step. It’s not just what you type in the prompt — it’s everything the model sees: system instructions, conversation history, retrieved code, tool descriptions, rules files, and metadata.

Prompt Engineering vs. Context Engineering

Prompt engineering is what you say. Context engineering is what you show. A perfectly crafted prompt with poor context produces poor results. A mediocre prompt with excellent context often produces great results. Research confirms: context quality matters more than prompt wording.

The Eight Context Layers

// Everything in the context window: 1. System prompt Agent personality & rules 2. Rules files Project-specific instructions 3. Retrieved code Files, snippets, types 4. Tool descriptions What the agent can do 5. Conversation Chat history & tool results 6. Few-shot examples Patterns to follow 7. Memory Summaries of past context 8. User prompt Your actual instruction // You directly control layers 2, 6, and 8. // You indirectly influence 3, 5, and 7. // Layers 1 and 4 are set by the tool.

Key insight: When AI gives bad results, most people blame the model or rewrite their prompt. The real fix is usually in the context: the model didn’t have the right code, types, or conventions visible. Context engineering is the highest-leverage skill for AI-assisted development.

description

Rules Files: Your Project’s Persistent Memory

AGENTS.md, CLAUDE.md, .cursorrules — the files that make agents smarter

What Rules Files Do

Rules files are plain Markdown documents injected into the system prompt at the start of every session. They answer the question: “What does an agent need to know upfront to enter this codebase safely and make reasonable decisions?” They bridge project memory, coding standards, team norms, and architectural decisions.

The Ecosystem

• AGENTS.md — Cross-tool standard (Linux Foundation). Read by Copilot, Cursor, Codex, Windsurf, Devin, Amp
• CLAUDE.md — Claude Code’s native format. Supports global, project, and per-folder hierarchy
• .cursor/rules/ — Cursor’s format. Supports file-pattern matching and glob-based activation
• .github/copilot-instructions.md — GitHub Copilot’s project instructions

What to Put in Rules Files

// Good rules file content: Tech stack Next.js 15, TypeScript, Prisma, Tailwind Test runner vitest, run with: npm run test Conventions camelCase variables, PascalCase components Forbidden Never use any, never disable eslint rules Architecture Feature-based folder structure, not layer-based DB schema Users have roles: admin, editor, viewer API pattern All endpoints return { data, error, meta } // Bad rules file content: Obvious “Write clean code” (too vague) Huge 500+ lines (consumes too many tokens) Stale Outdated conventions nobody follows

Impact: Projects with well-maintained rules files produce 1.7x fewer defects and 2.74x fewer security vulnerabilities compared to ad-hoc AI coding. The investment in writing good rules pays compound returns on every session.

search

How Agents Find Code: Search & Retrieval

Grep, semantic search, and codebase indexing working together

Three Search Strategies

Exact search (Grep) finds code by literal text or regex. Fast, precise, but only works when you know the exact symbol name. Best for: finding function definitions, import paths, error messages.

Semantic search finds code by meaning, not text. Uses vector embeddings to match queries like “where do we handle authentication?” even if the word “authentication” never appears. Best for: exploring unfamiliar code, finding related logic.

AST-based search parses code into syntax trees and searches by structure. Can find all functions that return a Promise, or all classes implementing an interface. Best for: structural refactoring, type-aware queries.

Codebase Indexing

Modern tools index your entire codebase into vector embeddings. Code is split into meaningful chunks (functions, classes, logical blocks), each converted to a vector. When the agent needs context, it searches these vectors for the most relevant chunks. Indexes sync automatically every few minutes.

Hybrid Retrieval

The best results come from combining strategies. Semantic search finds broadly relevant files, then exact search pinpoints specific symbols within them. Research shows hybrid retrieval produces 12.5% higher accuracy on codebases with 1,000+ files compared to either strategy alone.

Your role: Agents discover context automatically through search tools, but you can accelerate this. Using @ mentions to reference specific files, pasting error messages, or saying “look at the User model in db/schema.ts” gives the agent a head start instead of making it search blindly.

tune

The Lost-in-the-Middle Problem

Why more context isn’t always better context

The U-Shaped Attention Curve

LLMs pay the most attention to information at the beginning and end of the context window, and significantly less to information in the middle. Performance degrades by more than 30% when critical information sits in the middle of a long context. This is an architectural property of how attention and positional encoding work — not a bug that will be fixed soon.

The False Security of Large Windows

Models now support 128K–1M+ tokens. This tempts people to dump entire codebases into context. But larger windows create problems: costs scale linearly, latency increases, and accuracy drops from signal-to-noise dilution. A 200K-token context with 5K tokens of relevant code means the model is searching through 195K tokens of noise.

Curation Strategies

// Position critical info at boundaries: START System prompt, rules, key types MIDDLE Supporting context, examples, history END User prompt, most relevant code // Keep signal-to-noise ratio high: DO Include only files relevant to the task DO Trim large files to relevant functions DO Use line ranges, not entire files DON’T Dump entire directories into context DON’T Include test files unless testing DON’T Add “just in case” context

Key insight: Context engineering is fundamentally a curation problem, not a capacity problem. The skill isn’t fitting more into the window — it’s choosing what deserves to be there. Every irrelevant token dilutes the signal the model needs.

alt_route

Manual Context Techniques

What you can do right now to give agents better context

@ Mentions & File References

Most tools support @ mentions to explicitly include files, folders, or symbols in the context. Instead of hoping the agent finds the right file, point it directly: @src/auth/middleware.ts. This is faster and more reliable than letting the agent search.

Open the Right Tabs

For completion-mode tools, open tabs are context. If you’re implementing a function that uses types from another file, open that file. If you’re following a pattern from an existing component, open that component. The tool will include these in the prompt automatically.

Paste Examples

The most powerful context technique: paste an example of what you want. If you need a new API endpoint, paste an existing endpoint and say “follow this pattern.” Few-shot examples in the conversation are worth more than paragraphs of description.

Structured Task Descriptions

// Low-context request: “Add a delete endpoint” // High-context request: “Add a DELETE /api/users/:id endpoint. Follow the pattern in @src/routes/posts.ts. Use the User model from @src/db/schema.ts. Require admin role (check auth middleware). Return 204 on success, 404 if not found. Add a test in @tests/users.test.ts.” // Same task. Dramatically different results. // The second version gives the agent: // - Pattern to follow (existing endpoint) // - Data model (schema file) // - Auth requirements (middleware) // - Expected behavior (status codes) // - Test location (test file)

The 30-second rule: Spending 30 seconds writing a detailed request with file references saves 5–10 minutes of back-and-forth corrections. The time investment in context always pays off.

account_tree

Automated Context: Graph-Based Retrieval

How tools automatically find the code your agent needs

Code Graphs

Advanced tools build a graph of your codebase tracking relationships between functions, classes, modules, and files. When you ask the agent to modify a function, the graph automatically pulls in: the function’s callers, the types it uses, the tests that cover it, and the files that import it. No manual @ mentions needed.

Session Memory

Graph-based tools also track session context: what files you’ve viewed, what decisions you’ve made, what errors you’ve encountered. This session memory means the agent doesn’t ask you to repeat information — it remembers that you decided to use JWT tokens three turns ago.

The Impact

Benchmarks show automated graph-based context retrieval produces significant improvements over manual context injection:

• +14 percentage points higher task completion
• 22% faster execution time
• 58% lower API costs
• 65% fewer tokens consumed per task

The system retrieves exactly what’s needed, nothing more. It’s better at curation than most humans because it can analyze the entire dependency graph instantly.

The trend: Context engineering is gradually shifting from a manual skill to an automated capability. But understanding how context works still matters — you need to recognize when the automation fails and know how to supplement it manually.

warning

Context Anti-Patterns

The mistakes that make AI coding sessions frustrating

The Kitchen Sink

Dumping your entire codebase or a huge directory into context. The model drowns in irrelevant code and misses the important parts. Fix: Reference only the specific files and functions relevant to the task.

The Blank Slate

Giving a one-line instruction with no context at all: “fix the bug.” The agent has to spend multiple turns searching for what you already know. Fix: Include the error message, the file, and what you expect to happen.

The Marathon Session

Working in a single session for hours on multiple unrelated tasks. Context from task 1 pollutes task 5. Compaction loses critical details. Fix: Start fresh sessions for distinct tasks. Keep sessions focused.

The Stale Rules File

A rules file written six months ago that references deprecated patterns, old dependencies, or conventions the team abandoned. The agent follows outdated instructions confidently. Fix: Review rules files monthly. Delete rules that no longer apply.

The Conflicting Context

Rules file says “use Prisma ORM” but the referenced example uses raw SQL. The agent gets confused and produces inconsistent code. Fix: Ensure rules, examples, and referenced code all agree. Audit for contradictions.

The root cause: Nearly 65% of AI agent failures in production come from context drift and misalignment — not from model limitations. When the agent produces bad code, ask: “Did it have the right context?” before blaming the model.

verified

The Context Engineering Mindset

Thinking like a context engineer changes everything

Before Every Request, Ask Yourself

// The Context Checklist: 1. Does the agent know what I want? → Clear task description with expected outcome 2. Does it know where to look? → File references, @ mentions, directory hints 3. Does it know how I want it done? → Pattern examples, conventions, rules file 4. Does it know what not to do? → Constraints, forbidden patterns, edge cases 5. Does it have the types and schemas? → Data models, interfaces, API contracts

The Compound Investment

Context engineering compounds over time:

• Week 1: Write your first rules file. 30 minutes.
• Week 2: Add conventions you keep correcting. 10 minutes.
• Week 4: Rules file handles 80% of your preferences automatically.
• Month 2: New team members get AI that already knows your codebase.
• Month 6: Your rules file is the best onboarding document you have.

The Transferable Skill

Context engineering transfers across every AI tool. When you switch from one tool to another, your understanding of context windows, retrieval, curation, and rules files applies immediately. Tools change every 6 months. Context engineering is the skill that lasts.

Key insight: The best AI-assisted developers aren’t the ones who write the cleverest prompts. They’re the ones who maintain the best context: clean rules files, well-organized codebases, thoughtful file references, and focused sessions. Context engineering is the new software craftsmanship.

Ch 7 — Context Engineering