Context Rot: Why Your AI Agent Gets Dumber on Long Tasks

Anyone who has run an AI agent on a long job has watched the same movie. The first hour is brilliant. It reads the codebase, files the tickets, writes clean summaries, anticipates the next step. Then somewhere around hour three it starts forgetting things it knew at hour one. It re-asks a question already answered. It reintroduces a bug it just fixed. The agent that opened sharp closes confused, and nobody told it to get worse.

That decay has a name now, and a mechanism. Anthropic calls it context rot, and it runs deeper than any quirk of one model or one bad prompt. It's physics.

The attention budget is finite

Here's the part most people building with AI never internalize. A transformer doesn't read a context window the way you skim a page. Every token attends to every other token, which means attention scales with the square of the token count, n² pairwise relationships across the whole window (Anthropic, Effective context engineering for AI agents, 2025). Double the context and you roughly quadruple the relationships the model has to hold in its head at once.

The model was trained on a finite pool of attention, and that pool gets spread thinner with every token you add. So as the window fills, the model's recall of any single item drops. Not all at once. It slides down a gradient instead of falling off a cliff. The agent doesn't crash at some magic line count. It just gets a little vaguer, a little less precise, a little more likely to lose the thread, and the more you stuff in to help it, the worse you can make it.

Think of it as a desk, not a filing cabinet. A filing cabinet holds more the bigger it gets. A desk gets harder to work on the more you pile onto it, even when every sheet on it is something you genuinely need. The context window is a desk.

Past prompt engineering

A couple of years ago the craft was prompt engineering: write the clever instruction, get the better answer. That worked when the whole interaction was one question and one reply. It doesn't describe what a modern agent actually does, which is run for hundreds of turns, calling tools, reading files, accumulating history.

Anthropic frames the next discipline as context engineering: curating the entire token set the model sees, live, across many inference turns. That means far more than the prompt. The system instructions, the tool definitions, the external data pulled in, the message history piling up behind the agent as it works. The guiding rule is almost austere. Find the smallest set of high-signal tokens that gets the outcome you want. Every token you add to help the model is a token competing for the same shrinking budget. Curation beats accumulation.

That reframe changes the job. You stop asking "what's the perfect prompt" and start asking "what does the model need in front of it right now, and what can I get off the desk."

The moves that keep agents sharp

The good news is the failure mode is well understood, and the fixes are concrete. Five of them carry most of the weight on long-horizon work.

Compaction. When history gets long, summarize it and restart the agent from the compressed version. Keep the load-bearing decisions, the architecture choices, the unresolved threads. Drop the redundant tool output nobody needs again. The instinct is to maximize recall first, then cut the dead weight, so you don't compress away the one fact the next step depends on.

Structured note-taking. Give the agent persistent memory outside the window: a NOTES file, a running to-do list it owns. Now state survives a context reset. The agent can have its desk cleared and still pick up where it left off, because the important things were written down somewhere that doesn't decay.

Sub-agents. Instead of one agent dragging an ever-heavier window through a sprawling task, spin up focused sub-agents that each work in a clean window and report back a tight summary, on the order of one to two thousand tokens (Anthropic, 2025). The lead agent stays light. The deep work happens in fresh context and comes back condensed.

Just-in-time retrieval. Don't pre-load everything the agent might conceivably need. Hold lightweight identifiers, file paths, URLs, record IDs, and load the actual data at runtime when the moment calls for it. It mirrors how people work. You don't memorize the whole filing system. You remember where things live and go get them.

Right-altitude prompts. The system prompt has a sweet spot. Too low and it's brittle, hardcoded if-this-then-that logic that shatters the first time reality doesn't match. Too high and it's vague inspirational mush the model can't act on. Right altitude is specific enough to steer behavior while leaving the model its own strong heuristics. It's the same skill as briefing a sharp new hire: enough to point them, not so much you're doing their thinking for them.

Why this is the real skill now

The teams that ship reliable agents win on something other than the cleverest single prompt. They treat context as the scarce resource it is and engineer around the n² ceiling instead of pretending it isn't there. They compact. They externalize state. They fan work out and pull it back in summaries. They load data the moment it's needed and not a turn sooner.

Context rot is no bug to wait out. It's the operating condition of every long-running agent, and it rewards the builders who design for a desk that fills up. The model's attention is the budget. Spend it like one.