Every tool call in a ReAct agent adds to the context: the model’s reasoning, the action taken, and the tool’s result. Each tool result can be hundreds or thousands of tokens — API responses, file contents, search results, error traces. A 10-step agent task can easily consume 30,000–50,000 tokens of accumulated action history alone.

Without intervention, the accumulated history fills the context window and pushes out the system instructions, tool definitions, and early task context that the model actually needs to reason well. The model loses access to its own instructions while drowning in the history of its own actions.

The simplest approach: keep only the most recent N turns of interaction and discard the rest. Fast and predictable, but brutal — critical early decisions, error patterns, and task context are permanently lost. The model has no memory of what it already tried.

Keep the first few turns (task setup) and the last few turns (recent context), discard the middle. Better than pure recency, but still loses the narrative thread of how the agent arrived at its current state.

The field has converged on sliding window plus summarization hybrids as the dominant compression approach. The pattern: keep the most recent N turns in full detail (the “window”), and compress older context through LLM-based summarization. The summary preserves key decisions, outcomes, and error patterns while dramatically reducing token count.

Compression can be triggered periodically (every N turns), threshold-based (when context exceeds X% of the window), or event-based (after a task phase completes). Periodic compression is simplest; threshold-based is most efficient. Each compression step requires an LLM call, so amortizing the cost by compressing periodically rather than every turn is standard practice.

Manus discovered a subtle but important finding: keep the most recent tool calls in raw format so the model maintains its “rhythm” and formatting style. When recent tool interactions are summarized instead of kept raw, the model loses its formatting consistency — it starts producing outputs in slightly different structures, breaking downstream parsers and tool integrations.

Manus’s second critical finding: do not compress away error traces. When a tool call fails, leaving the error and stack trace in context helps the model avoid repeating the same mistake. This technique is well-established — libraries like Instructor use it for structured output retries — and it applies broadly to any agent that calls tools.

Instead of compressing older turns into a summary, the long-term memory approach moves them to a durable storage system (typically a vector database) and retrieves only relevant actions on demand. The context window carries only the recent window plus any retrieved historical context that’s relevant to the current task.

Working memory: Active context window — immediate reasoning.

Recall storage: Searchable database of recent interactions — retrieved when relevant.

Archival storage: Vector-based long-term memory — accumulated knowledge across all sessions.

Prompt caching is a server-side feature (available in Anthropic, Groq, and vLLM) that caches the KV states for shared prefixes across requests. If your system prompt and tool definitions are the same across requests, they’re computed once and reused. This provides up to 90% cost savings on repeated content and is the most impactful “compression” technique for the stable prefix.

An emerging technique: test-time training temporarily fine-tunes the model on the current context, achieving a 35× speedup for 2M-token contexts compared to processing the full context directly. Still experimental, but it points toward a future where compression happens at the model level rather than the prompt level.

The most fundamental compression technique is also the simplest: don’t include what you don’t need. Before adding any content to the context, ask: does the model need this for the next step? If not, leave it out. This “Select, Don’t Dump” principle prevents bloat before compression is even needed.

Companies implementing effective context compression report 60–80% cost reduction on long-running agent tasks. At enterprise scale (thousands of agent tasks per day), this translates to hundreds of thousands of dollars in annual savings. The compression LLM calls add cost, but the net savings are overwhelmingly positive.

All compression is lossy. Summarization preserves the gist but loses details. The question is always: which details can you afford to lose? Compression works well for long-horizon tasks where early steps are contextual background, but poorly when critical early details get summarized away — like a specific error message or a user’s exact phrasing of a requirement.

The quality of the compression depends on the quality of the summarization model. Using a cheaper, faster model for compression saves money but may lose important nuance. Using the same model for compression and reasoning is more accurate but doubles the cost of each compression step.

When to compress? Too early loses detail; too late wastes tokens on bloated context.

What detail level? Aggressive compression saves more tokens but loses more information.

How to evaluate? Measuring whether a summary preserved “enough” information is itself an unsolved problem. Probe-based evaluation (asking the model questions about compressed content) is the current best practice.