Context rot: two guards, split by model, and one model gets neither

Quick answer

September 2026. Context rot is real, it is now named in Anthropic's own documentation, and it is not the thing you should be spending your afternoon on. The useful question is narrower: your Claude model ships with one of two self regulation mechanisms, or with neither, and they do not overlap.

Claude Sonnet 5, Sonnet 4.6, Sonnet 4.5 and Haiku 4.5 get context awareness automatically. The API tells them how much of the window is left, every turn, for free. Claude Opus 5, Opus 4.8, Opus 4.7 and the Fable and Mythos models get task budgets instead, which are beta, opt in, and do nothing unless you send a header. Claude Opus 4.6 gets neither.

So the cheaper models know how much room they have left, and the flagship does not unless you ask. That is the inversion worth knowing, and I got it the wrong way round for about a week.

The moment

I had a refactor agent that had been running fine on Sonnet for a month. Long loops, twenty to forty tool calls, a lot of file reading. I moved it to Opus for a job I thought needed the extra depth.

It got worse. Not at the start, at the tail. Around the point where a session had eaten most of its room, the Sonnet version used to start consolidating: shorter tool calls, a summary, a clean stop. The Opus version kept opening new threads of work. It proposed a schema change with maybe eight thousand tokens of headroom left, then ran out mid sentence.

I spent two days on the prompt. The prompt was not the problem. On Sonnet, the model had been getting a running countdown of its remaining context injected into every turn, and it had been quietly pacing itself against it. On Opus, nothing was being injected, and I had not sent the header that would have given it the equivalent. It was not reasoning worse. It was reasoning blind.

Finding 1: the vendor now uses the community's term

"Context rot" was coined by a Hacker News commenter in June 2025, and for a year it was a folk term. It is now in the platform docs. Anthropic's context windows documentation states it plainly:

Anthropic, context windows documentation, read September 2026: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot. This makes curating what's in context just as important as how much space is available."

The same page opens with "more context isn't automatically better". That is a vendor documenting a limitation of its own headline number, which is rare enough to be worth reading twice.

Finding 2: the study everyone cites tested models you are probably not running

Almost every context rot article traces back to one piece of work: Chroma's technical report on input length and LLM performance. It is good, it is replicable, and it evaluated 18 models.

Here is the part nobody repeats. The Claude models in that study were Opus 4, Sonnet 4, Sonnet 3.7, Sonnet 3.5 and Haiku 3.5.

In September 2026 the current line is Opus 5, Sonnet 5 and Haiku 4.5. Every Claude model in the canonical study is at least a generation behind what you are calling today, and the 1M token window those newer models ship with did not exist in the tested configuration.

I am not saying the findings are void. They are still the best public evidence for the shape of the problem, and Anthropic clearly agrees, because it links the same conclusions from its own docs. I am saying that if you are quoting a specific degradation curve at someone, year tag it and name the model, because the curve you are quoting was measured on hardware nobody is shipping.

Finding 3: shuffling the haystack made things better

The counterintuitive result, and the one I keep reteaching people:

Chroma, context rot report: "models perform worse when the haystack preserves a logical flow of ideas. Shuffling the haystack and removing local coherence consistently improves performance."

Read that again. A well organised, logically flowing document was harder for the models than the same content shuffled into incoherence. The instinct to tidy your context into a nice narrative before handing it over is not obviously correct.

I have no mechanism to offer for this and neither does the report. It is an observation, it replicated across models, and it should make you suspicious of any advice that amounts to "structure your context better" without saying what better means.

Finding 4: one distractor is enough, and they are not interchangeable

Two related results from the same work:

• A single distractor measurably degrades performance relative to a needle only baseline. Four distractors compound it. You do not need to fill the window with junk; a little relevant looking junk is enough.
• Distractors are not fungible. Specific ones consistently caused a bigger drop than others, across models. So "how much irrelevant context" is the wrong question. Which irrelevant context is the right one.

The modifier on top of both is needle question similarity: as the semantic similarity between the question and the answer drops, performance degrades faster as input grows. Easy lookups survive long context. Inference that requires connecting two things that do not share vocabulary is what falls apart first, and that is most real debugging.

Finding 5: it is a gradient, and that is worse than a cliff

A cliff would be manageable. You would find the edge, stay back from it, and write an assertion. Anthropic's effective context engineering guide is explicit that this is not what happens: the result is a "performance gradient rather than a hard cliff", with "reduced precision for information retrieval and long-range reasoning".

The stated causes are worth knowing because they explain why no threshold exists:

1. An attention budget. Finite working memory, spent across whatever you loaded.
2. Transformer scaling. Every token attends to every other token, so the pairwise relationships grow with the square of the length and the same attention is spread thinner.
3. Training distribution. Short sequences are far more common than long ones, so models have "fewer specialized parameters for context-wide dependencies".

None of those produce a number you can code against. They produce a slope. Which is exactly why the two runtime mechanisms below exist.

Finding 6: two guards, split by model, and one model gets neither

This is the finding I actually came for. Anthropic ships two different ways for a model to know how much room it has left, and which one you get is decided by your model string.

Context awareness is automatic. The API injects the total window into the system prompt and, after each tool call, a warning line reading Token usage: 35000/200000; 165000 remaining. Nothing to enable, nothing to send. Per the context windows docs it applies to Claude Sonnet 5, Sonnet 4.6, Sonnet 4.5 and Haiku 4.5, and the docs are equally explicit that Opus 4.7 and later Opus models, plus Fable 5.1, Mythos 5.1, Fable 5 and Mythos 5, "don't receive these injected tags".

Task budgets are the Opus side alternative, and they are beta. You send the header task-budgets-2026-03-13 and a task_budget object, and the model gets a countdown across the whole agentic loop.

output_config = {
    "effort": "high",
    "task_budget": {"type": "tokens", "total": 64000},
}

Now put the two support lists side by side, which no page I found does:

Three things fall out of that table.

The first is that the two mechanisms are mutually exclusive. There is no model in the list that has both. Sonnet 5 is explicitly "Not supported" for task budgets in Anthropic's own feature support table, and it does not need them, because it already gets the automatic version.

The second is that Claude Opus 4.6 has neither. It has a 1M token window and no self regulation signal by either route. If you are running long agentic loops on 4.6, the model genuinely does not know how much room is left, and there is no header that will tell it.

The third is the inversion I opened with. The default posture is better on the cheaper models. Sonnet and Haiku get the guard for free. On the flagship you have to know the feature exists and opt in, which means the most expensive long running loops are the most likely to be running blind.

Finding 7: the countdown is invisible to you

This one cost me an afternoon of trying to log something that does not exist.

Anthropic, task budgets documentation: "The countdown is visible only to the model. API responses do not include a remaining-budget field: there is no task_budget information in the response usage object, and SDKs have no accessor for it."

So you cannot log the remaining budget, alert on it, or assert against it in a test. If you want a client side figure you sum it yourself across the loop:

spent = sum(r.usage.output_tokens for r in loop_responses)

and accept that your number and the model's number are measuring different things, which brings us to the next one.

Finding 8: the budget counts what the model sees, not what you send

In an agentic loop your client resends the whole conversation every request, so the payload grows every time. The budget does not decrement by the payload. It decrements by what is new: tokens the model generates, plus content it has not seen before.

Anthropic's worked example makes the gap concrete. Across a three request turn the client transmits roughly 20,820 input tokens, and the budget counts 19,000. The first user message is sent three times and counted once.

The trap sits right next to it. If you decrement remaining yourself while also resending the full history, you double count, the model sees an under reported budget, and it wraps up earlier than it needed to. The docs name this failure directly and the advice is to set a generous budget once and let the server track it rather than mirroring it client side.

That is the tidy engineering instinct being wrong twice in one feature: once for shuffling the haystack, once for tracking your own budget.

Finding 9: a budget set too small looks like a refusal

The minimum accepted task_budget.total is 20,000 tokens and anything smaller returns a 400. Above that floor the failure mode is not an error at all:

Anthropic, task budgets documentation: "When Claude sees a budget that is clearly insufficient for the work being asked ... it may decline to attempt the task at all, scope it down aggressively, or stop early with a partial result rather than start work it cannot finish."

A model declining to start reads exactly like a capability problem or a safety refusal. It is neither. If you set a budget and then see unexplained refusals or premature stops, raise the budget before you touch anything else. Note also that task budgets are advisory, described as a "soft hint, not a hard cap". The only enforced ceiling is still max_tokens.

Finding 10: caching does not buy you context room

Worth stating because I have heard it twice this month. Prompt caching does not make cached tokens stop occupying the window:

Anthropic, context windows documentation: "Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count."

Caching is a cost and latency feature. It is not a context management feature, and a cached 200k prefix rots exactly as much as an uncached one. If you want the window back, that is compaction or context editing, which is a different lever, and one I have written about from the auto compact side already.

There is a second order interaction if you use both: the budget countdown marker is injected server side per request, so mutating the budget client side changes the rendered prompt and invalidates the cache prefix that contained it.

Finding 11: Claude Code does not get task budgets

Since most of what I write here is CLI shaped, the boundary matters. Anthropic's support table states that task budgets are "not supported on Claude Code or Cowork surfaces", and directs you to use them through the Messages API on a supported model.

So inside Claude Code the picture is the Sonnet or Opus split from Finding 6 with the opt in half removed. Which is a reasonable part of why the same long session feels different on the two model families, and it is a separate question from how big the context window is.

What I changed

Three things, none of them clever.

1. Long agentic loops on Opus now send a task budget. One header, one object, sized from the p99 of my own measured per task spend rather than a round number.
2. I stopped tidying context into narrative order. Given the shuffled haystack result I no longer spend effort arranging retrieved chunks into a flowing document. I spend it on removing chunks instead, which the distractor result says matters more.
3. I treat an unexplained early stop as a budget symptom first. It was a prompt problem exactly zero times out of the four occasions I have now chased it.

What I did not verify

• I did not run my own long context evaluations. Every degradation figure here is Chroma's or Anthropic's, and I am reporting them, not reproducing them.
• I did not measure whether Opus 5 rots less than the Opus 4 in the study. Nobody has published that comparison as far as I can find, which is precisely the gap that makes the canonical study awkward to quote in 2026.
• I could not observe the injected context awareness tags directly. The docs say the API injects them and that you never send them yourself; they are not in any response I can read, so that part is documentation, not measurement.
• I tested task budgets on Opus 5 only. The support table lists seven models and I exercised one of them.
• I have not measured the cost of the header. Whether turning on a task budget changes output quality on an otherwise identical task is a real question and I do not have an answer worth publishing.

Postscript: the whole reason this was hard to diagnose is that a model with no budget signal does not behave like a model that is failing. It behaves like a model that is confident.