Since 2023, a wave of frameworks has emerged for orchestrating multiple LLM-powered agents: Microsoft’s AutoGen, open-source patterns like CrewAI, LangGraph for stateful agent graphs, and many others. They share a core idea: wrap LLM calls in agent abstractions with roles, tools, and conversation protocols. The differences lie in how much structure they impose (free chat vs rigid DAG), human-in-the-loop support, and observability. This chapter surveys the design patterns rather than endorsing a single library.

The simplest multi-agent pattern: assign each agent a distinct system prompt that defines its role, expertise, constraints, and output format. A “Planner” agent decomposes tasks; a “Coder” writes code; a “Reviewer” critiques. Role clarity reduces scope creep (one agent trying to do everything) and makes failures attributable. Keep roles narrow and testable — if you cannot write a unit test for a role’s expected behavior, the role is too vague.

LLM agents become useful when they can call tools: search APIs, databases, code interpreters, file systems. Frameworks provide tool registries mapping function names to implementations. Key design choices: which agents get which tools (least privilege), approval gates for destructive tools (delete, pay, deploy), and retry/fallback when tools fail. Log every tool call with inputs, outputs, latency, and cost — this is your audit trail and debugging lifeline.

Round-robin: agents take turns in a fixed order — simple, predictable, but can waste turns. Debate: two agents argue opposing positions; a judge picks the winner — good for reducing hallucination. Reflection: an agent critiques its own output before passing it on. Hierarchical: a manager agent delegates to specialists and synthesizes. Mix patterns: use debate for high-stakes decisions, round-robin for brainstorming, and hierarchical for execution. Always cap max turns.

Fully autonomous multi-agent systems are risky for high-stakes tasks. Human-in-the-loop (HITL) patterns: approval gates before tool execution, review checkpoints after planning, escalation when confidence is low or agents disagree. Design the UX so humans see a summary + diff, not a wall of agent chat. Track approval latency — if humans are the bottleneck, the system needs better defaults or tighter scoping, not more autonomy.

Agents need short-term memory (current conversation context), long-term memory (past interactions, learned facts), and shared state (team knowledge base). Frameworks offer vector stores for semantic retrieval, key-value stores for structured facts, and conversation buffers with summarization. Pitfalls: stale memory (outdated facts never evicted), context overflow (stuffing too much into the prompt), and privacy leaks (Agent A reading Agent B’s private memory).

Multi-agent systems are hard to debug: failures cascade, blame is distributed, and logs are interleaved. Essential observability: trace IDs linking all messages in a task, agent-level metrics (tokens, latency, tool success rate), conversation replays with timestamps, and cost attribution per agent. Use structured logging (JSON) and build dashboards that show the critical path through the agent graph. Without this, production issues become archaeology.

Evaluate frameworks on: conversation control (can you enforce your protocol?), tool integration (registry, approval gates), human-in-the-loop UX, observability (traces, cost tracking), state management (memory, persistence), and community/maintenance. Start with the simplest pattern that works (often one planner + one executor). Add agents only when you have clear role boundaries and measurable improvement. Next chapter: how to evaluate all of this rigorously.