The link in your system prompt is the one Claude will not fetch

Quick answer

September 2026. Claude's web_fetch tool decides what it is allowed to retrieve by asking where the URL came from, not what the URL is. A link that appears only in your system prompt is not fetchable. A link Claude wrote itself is not fetchable. A link that came back from code execution, the MCP connector or tool search is not fetchable either.

But a link that came back from one of your own client-side tools is fetchable, even when that tool is echoing text Claude produced a moment earlier.

That last pair is the part worth internalising, and it is why my first attempt failed in a way that looked like a domain-filtering problem and was not. The allowlist is about provenance. The same string is reachable or unreachable depending on which kind of tool surfaced it.

Second thing, unrelated but it caught me on the same afternoon: there are now four versions of this tool, not one, and they are capability-keyed rather than a ladder.

The moment

I had a small summarising job. A system prompt with a list of approved documentation URLs, a user asking a question, and web_fetch enabled so the model could go and read whichever of those pages was relevant.

It never fetched anything. It would say, politely, that it did not have access to the page.

I spent an embarrassing amount of time on allowed_domains. I added the hosts. I removed the hosts. I checked for a typo in the scheme. I re-read the domain rules twice. Then I moved one URL out of the system prompt and into the user turn as a test, and it fetched immediately.

The tool had been behaving correctly the entire time. I had simply put the URLs in the one place the tool is documented not to look.

Finding 1: the allowlist is about provenance, not the URL

The rule, quoted from Anthropic's web fetch documentation:

Verbatim: "For security reasons, the web fetch tool can only fetch URLs that have previously appeared in the conversation context."

Three sources count:

• URLs in user messages
• URLs in client-side tool results
• URLs from previous web_search or web_fetch results

This is a data-exfiltration control, and once you read it that way the behaviour stops being surprising. If the model could fetch any URL it emitted, then anything it had read could be encoded into a hostname or a path and sent somewhere. Restricting fetches to URLs that something other than the model put on the table closes most of that door.

The mental model that actually works: a URL is not a string here, it is a string plus a history. Two identical URLs can have different fetchability in the same conversation.

Finding 2: your system prompt is not part of the conversation

This is the one that cost me the afternoon, and it is stated plainly once you know to look for it.

Verbatim: "The tool cannot fetch URLs that appear only in Claude's own output or only in the system prompt. To make a URL from the system prompt fetchable, also include it in a user message."

The documented fix is duplication. You put the URL in both places.

That feels wrong the first time, because a system prompt is the most trusted thing in the request. But trust is not what the rule is keyed on. The rule is keyed on whether the URL arrived in the turn-by-turn record, and a system prompt sits outside it.

If you build a system prompt from a template that injects a documentation index, this will bite you, and it will bite you silently, because the model will produce a plausible answer from its own knowledge rather than an error you can grep for.

Finding 3: server-tool output does not count, client-tool output does

Here is the asymmetry I did not expect.

Verbatim: "Results of other server-side tools, such as code execution, the MCP connector, or tool search, are not an allowed source either. Client-side tool results are an allowed source even when they echo text that Claude produced (for example, a command that prints its input, or an error message that quotes it)."

Read those two sentences next to each other. A URL that comes out of Anthropic-hosted code execution is blocked. A URL that comes out of a tool running on your machine is allowed, and the docs explicitly say this holds even when your tool is just parroting back something Claude wrote.

The logic is about who is vouching. A client-side tool result is your application asserting something. A server-side tool result is still, in a sense, the model's own loop talking to itself, so it does not reset the provenance clock.

The practical consequence is concrete. If your pipeline discovers URLs by running code (scraping a sitemap, parsing a JSON index, reading a file), those URLs are dead to web_fetch. Two ways out:

1. Do the discovery with web_search, whose results are an allowed source. The docs describe this combined pattern directly, and it is why the two tools are usually declared together.
2. Move the discovery step into a client-side tool so your own code hands the URL back.

Declaring both, which is the shape I ended up with:

{
  "tools": [
    { "type": "web_search_20260318", "name": "web_search", "max_uses": 3 },
    {
      "type": "web_fetch_20260309",
      "name": "web_fetch",
      "max_uses": 5,
      "use_cache": false,
      "max_content_tokens": 50000,
      "citations": { "enabled": true }
    }
  ]
}

Note citations there. It is optional on web_fetch and disabled by default, which is the opposite of web_search, where citations are always on and not configurable. If you are rendering fetched content back to a user, that default is not the one you want.

It is also worth saying the quiet part: this is a strong control, not a perfect one. Anthropic's own warning says there is "still residual risk that you should carefully consider when using this tool." Simon Willison wrote about exactly this trade-off when the tool first shipped, and his framing still holds up a year on: allow-listing domains you fully control is the version of this you can actually reason about.

Finding 4: there are four versions, and the newest is not automatically the right one

When I started, I assumed a version string was a date stamp and you took the latest. That is not how these are organised.

The tool reference classifies this as capability-keyed, and spells out the consequence:

Verbatim: "In each case, both the new and old versions are current; which one you use depends on whether you need the new capability."

So this is not a deprecation ladder and it is not model-keyed the way the text editor tool versions are. Older versions are not stale. Pick the lowest version that has the capability you need, and you inherit a smaller surface.

web_search has its own three-version set on the same pattern, which is easy to mix up when you declare both tools in one request.

Finding 5: caching is on by default, and on two of the four versions you cannot turn it off

use_cache defaults to true.

Verbatim: "The web fetch tool caches results to improve performance and reduce redundant requests. The content returned may not always reflect the latest version available at the URL."

If you are fetching a changelog, a status page, a pricing page, or anything else that moves, the default is working against you. Setting "use_cache": false fixes it, and that parameter requires web_fetch_20260309 or later. On 20250910 and 20260209 there is no way to ask for fresh content at all.

The docs also push back on doing it reflexively, which I think is fair: bypassing the cache costs latency, so it is for sources that genuinely move rather than a default you flip everywhere.

Finding 6: failures arrive as HTTP 200

Verbatim: "When the web fetch tool encounters an error, the Claude API returns a 200 (success) response with the error represented in the response body. Claude sees the error result and continues the turn."

Nothing raises. Your SDK does not throw. The turn completes and the model carries on, usually by answering from memory, which is precisely the failure that looks like success.

The error codes worth knowing:

• url_not_in_prior_context: the provenance rule above, by name
• url_not_allowed: domain filtering, or an Anthropic-side restriction such as a private address or robots.txt
• url_too_long: the ceiling is 250 characters
• unsupported_content_type: only text, HTML and PDF
• max_uses_exceeded, too_many_requests, url_not_accessible, invalid_tool_input, unavailable

If you are running this in anything resembling production, walk the response content for web_fetch_tool_result blocks and check whether the inner type is the error variant. The success and error shapes differ, so branch before you index into them.

One more that is easy to miss: failed fetches count against max_uses. A run that trips a domain rule five times can exhaust a budget of five without having read anything.

Finding 7: dynamic filtering silently brings code execution with it

Verbatim: "Dynamic filtering runs on the code execution tool, which the API enables automatically for the request. You don't need to add the code execution tool to the tools array."

So on 20260209 and later, declaring one tool enables two. Adding code_execution yourself as well is the documented mistake, and the cost of it is a second execution environment that the model then has to choose between.

Finding 8: the platform gaps are larger than for most tools

Web fetch is available on the Claude API, Claude Platform on AWS, and Microsoft Foundry. It is not currently available on Amazon Bedrock or Google Cloud at all. On Foundry deployments hosted on Azure, only the basic web_fetch_20250910 works, without dynamic filtering.

If you are writing provider-portable code, this is not a feature-flag difference, it is a whole tool that is missing on two of the major routes.

Finding 9: on Managed Agents the vocabulary is identical and the rules are not

Same allowed_domains and blocked_domains names, different constraints:

• A web_fetch domain cannot include a path. A web_search domain can carry one such as example.com/blog. Same two words, two different grammars, in the same config array.
• Each list is capped at 64 domains.
• Domains must be ASCII, so internationalised names need the Punycode form. The Messages API accepts Unicode and merely recommends against it.
• max_uses, citations and cache_control are not available on the agent toolset at all.

And the trap that is purely about reading comprehension: www.example.com does not cover example.com. A leading www. is a subdomain like any other. List the bare domain if you want both.

What I changed

Three things, all small.

I moved the documentation URLs out of the system prompt and into a user turn. The system prompt still describes what they are for; the user turn carries the actual links. Duplication, exactly as documented, and it works.

I pinned web_fetch_20260309 rather than the newest, because cache bypass is a capability I need for changelog pages and response inclusion is not. Lowest version that does the job.

I added an explicit check for the error variant of web_fetch_tool_result instead of assuming a 200 meant a fetch. That one found two failures that had been quietly answered from model memory for about a week.

The token side of this deserves more care than I gave it. A large documentation page is roughly 25,000 tokens, and a research PDF can be 125,000. max_content_tokens is the lever, it is approximate by the docs' own admission, and it does not apply to binary content such as PDFs. That interacts badly with everything in the context-rot notes from yesterday, and I have not finished thinking about it.

What I did not verify

Honestly, more than I would like.

• The docs list dynamic filtering support for Fable 5.1, Mythos 5.1, Fable 5, Mythos 5, Mythos Preview, Opus 4.8, Opus 4.7, Opus 4.6, Sonnet 5 and Sonnet 4.6. Claude Opus 5 is not in that list. I did not test whether that is a genuine gap or a documentation omission, and I am not going to assert which it is.
• I did not test the client-side-tool-echo case adversarially. The docs say a client tool result counts even when it echoes Claude's own text; I read that, I did not try to abuse it.
• I did not measure the cache TTL. The docs say the behaviour is managed automatically and may change, so I would not trust a number I derived anyway.
• I did not test the 250-character URL ceiling against a real long URL. I took the documented figure.
• Everything about Managed Agents domain rules here is read from documentation. I have not run an agent against a deliberately malformed domain list to see the error text.

Postscript: the fix for my original problem was to type the same URL twice, which is the least satisfying afternoon-ending discovery I have had in a while.