The list of bots that support Markdown Content Negotiation

What is Markdown Content Negotiation?

Markdown Content Negotiation is a way for a bot to ask the same URL for a Markdown version of a page, instead of receiving regular HTML.

This article tracks which bots and AI agents actually do that.

The mechanism is content negotiation: the bot sends an HTTP request with an Accept header that tells the server which format it prefers. In this case:

Accept: text/markdown

That sounds technical, but the distinction matters. This is not about publishing .md files, not about llms.txt, and not about a tool that downloads HTML and converts it to Markdown on its own side. I am checking a narrower question here: who asks for Markdown at request time, and who still gets HTML as usual.

Markdown support table

Bot or tool	Official support	Details
Claude Code / WebFetch Coding agent	Yes	The documentation says WebFetch sends a User-Agent that starts with `Claude-User` and an `Accept` header that prefers Markdown over HTML. Source: Claude Code documentation
GitHub Copilot CLI / web_fetch Coding agent	Yes	The changelog says `web_fetch` prefers Markdown when available, using HTTP content negotiation. Source: GitHub Copilot CLI changelog
OpenCode / webfetch Coding agent	Yes	The default is `format: markdown`, and the tool builds an `Accept` header that prioritizes `text/markdown`. Source: OpenCode source
Gemini CLI Coding agent	Yes direct mode	In the direct/experimental path, the tool sends an `Accept` header that prefers `text/markdown`. The regular Web Fetch documentation describes use of `urlContext`. Source: Gemini CLI source
Cursor Coding agent	No
GPTBot OpenAI crawler	No
OAI-SearchBot OpenAI crawler	No
ChatGPT-User User-triggered fetch	No
ClaudeBot Anthropic crawler	No
PerplexityBot Perplexity crawler	No
Googlebot Google Search crawler	No

What counts as official support here

I mark a tool as “Yes” only when there is an official source: product documentation, an official changelog, or the project’s own source code. External measurements can be useful, but they are still external measurements.

The reason for the split is simple: a lot of the discussion around Markdown for bots mixes several different things:

A site returning Markdown when asked for Accept: text/markdown
A tool downloading HTML and converting it to Markdown by itself
A separate file such as llms.txt or standalone .md pages

Only the first one is real content negotiation on the same URL.

What the list says right now

As of August 22, 2026, the clear support is mostly in coding agents. That makes sense: Claude Code, Copilot CLI, OpenCode, and Gemini CLI read a lot of technical documentation, and Markdown is a cleaner, cheaper format for that work.

For answer-engine crawlers, the picture is different. There is currently no official Markdown Content Negotiation support for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, or PerplexityBot. That does not mean they will never support it. It does mean I would not build a technical SEO or GEO argument as if they already ask for the format.

What to do with this

For documentation sites, API docs, and developer centers, this is already interesting. These are exactly the pages coding agents read, and that is where there is real demand for a clean Markdown version.

For content sites, news sites, and marketing sites, this is still something to watch more than an emergency project. You can test it, and you can try it if your infrastructure already supports it, but I would not pay for “AI agent optimization” that comes down to one switch.

The basics still matter more: clean HTML, content that can be quoted, server-side rendering, and knowing which bots actually read your site.

Sources

Good luck.