What is user-triggered fetcher?

The difference between a user-triggered fetcher (fetcher for short) and a regular crawler is who pulled the trigger. Googlebot decides on its own when to crawl you. ChatGPT-User arrives because someone, right now, asked ChatGPT a question and the system decided the answer lives on your site. The request runs an errand for a real user, not for an engine.

How user-triggered fetchers work

A user-triggered fetcher acts on a user’s request: someone asked a question, the system decided the answer is on a specific page, and the request goes out to it now. And because the fetch goes out on a user’s behalf, the rule that confuses people most applies to it: robots.txt usually does not stop it. The reasoning is documented by the companies themselves: the file exists to restrain automated crawling, and a request a user initiated is not automated crawling. Google writes that its fetchers generally ignore the rules because the user requested the fetch, and OpenAI writes that robots.txt may not apply to ChatGPT-User. The exception is Anthropic, which states that Claude-User does honor robots.txt.

That is why blocking a fetcher that ignores robots.txt goes through the server, not through the file, and increasingly web-bot-auth, which Google is already trying on Google-Agent.

What user-triggered fetchers look like in server logs

This is how the three major fetchers identify themselves in the user-agent string:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) Chrome/W.X.Y.Z Safari/537.36

What identifies the bot is the token, ChatGPT-User/1.0 or Claude-User/1.0 or Google-Agent. The Mozilla prefix is browser decoration, which is exactly why generic WAF rules misfire, and Chrome/W.X.Y.Z in Google’s string is a literal placeholder, not a value to search for. A full breakdown of Google’s fetcher family, when each arrives and what to do about it, is in the Google bots guide. And each of those lines is almost always a user’s question that led to your page, and a chance to be cited in the answer.

How user-triggered fetchers relate to SEO and GEO

Fetcher requests are the moment of truth: the visit that happens one second before the answer gets written. A site that blocks AI-engine fetchers, deliberately or accidentally through a bad WAF rule, deletes itself from the answer at the exact moment a user asked. Every fetcher request is a potential citation, but not every citation goes through a fetcher: in ChatGPT, appearing in search answers is governed by OAI-SearchBot’s index, not by ChatGPT-User. That makes fetcher traffic in your logs the minimum of your AI visibility, not the full picture.

The difference between a user-triggered fetcher and a crawler

A crawler builds a long-term index, and respects robots.txt if it is an honest bot. A fetcher serves a user in real time and is usually exempt from it. When analyzing logs or writing blocking policy, separate the two: blocking a crawler removes you from tomorrow’s index, blocking a fetcher removes you from the answer being written right now.