llms.txt has become a fixture in GEO guides and AI-visibility checklists.
The recommendation shows up in nearly every list and every guide: add an llms.txt file so AI engines see you and cite you. By now it sounds like something you must have.
So before you add it, it’s worth asking whether the file actually does what it promises. At Wix I managed the team that built llms.txt support, and when I went to check the logs for who actually pulls this file, I found almost no one.
What is llms.txt?
llms.txt is a single text file that sits at the root of your site, at /llms.txt, just like robots.txt? What is robots.txt? robots.txt is a text file that says which URLs crawlers may fetch. robots.txt controls crawling, not indexing. Full definition of robots.txt . But robots.txt tells bots where they can’t go, and llms.txt tries to do the opposite: point language models at what matters.
It’s a list of links to your key pages that someone picked by hand, each with a title and a short summary. Not a copy of your site, and not a summary of it. The idea is simple: instead of a model crawling your whole site and guessing what’s relevant, it gets a ready-made map.
Who created llms.txt?
The proposal was published by one person, Jeremy Howard, in September 2024, and no engine has ever adopted it as a standard. The backdrop was the limited context window models had at the time: one small file concentrating the essentials was a way around the limit.
And the original proposal says who the file is for: a model fetching it the moment a user asks for help. Not training, and not showing up in answers nobody asked for.
What does an llms.txt file look like?
The file is written in Markdown, and the proposal lays out its structure: an H1 with the site’s name (the only required part), a short blockquote summarizing the site, and then link lists split under H2 headings. This is the skeleton the proposal itself gives:
# Site name
> One sentence summarizing what's here
## Docs
- [Quick start](https://example.com/start.md): what's covered and why it matters
- [API reference](https://example.com/api.md): the full reference
## Optional
- [Changelog](https://example.com/changelog.md)The Optional heading carries meaning in the proposal: those are the links a model may skip when context is short. And the proposal has a separate second part: a clean Markdown version of every page, at the same URL with .md appended.
In plenty of tools you’ll also meet a file called llms-full.txt, which packs the actual page content into one file instead of linking to it. That’s a convention that grew up in the tools later, not part of the proposal.
That’s the whole file. A list of links with descriptions.
Why does everyone recommend llms.txt?
The recommendations started back in 2024, with the documentation-site adoption wave. But the jump into SEO checklists came in May 2026, when Chrome added the Agentic Browsing category to Lighthouse, its built-in page-audit tool, still marked experimental. It includes a check for whether an llms.txt file exists, and a lot of people saw the new audit and assumed it was an SEO requirement.
But look where Chrome filed that check: under Agentic Browsing. Not under SEO.
What’s the difference between agentic and GEO?
There are two completely different things being mixed up here. Agentic AI? What is Agentic SEO? Agentic SEO is adapting a site for AI agents that browse and act on it for users, not just crawlers reading it. Full definition of Agentic SEO is agents taking actions: AI browsers, the Claude Chrome extension, and bots that navigate a site and do things on it.
GEO, AI visibility, is something else: getting your brand shown and cited in the answers of ChatGPT, Gemini and Perplexity.
llms.txt, if anything, belongs to the first world. To an agent that needs a clean map of the site to operate on it. There’s no proof it does anything for the second one, the one that decides whether you get cited.
And there’s a structural reason it can’t: an AI answer cites a page that answers a question. An index of links doesn’t answer anything, so there’s no reason for it to turn up in an answer. At best it’s a signpost to pages that have to be good on their own.
Does Google use llms.txt?
The answer, at least from Google, is no. Google added an explicit section to its official docs: you don’t need to create new files to appear in Google Search, because Google Search doesn’t use them. And it puts AEO and GEO in their place: from Search’s perspective, that’s still SEO, and the files those checklists ask for aren’t needed.
Sounds settled. Except I don’t take everything Google’s team says at face value.
Google speaks from the perspective of Google Search, and it scopes things narrowly. “Search doesn’t use llms.txt” is a fact about Google Search only. It says nothing about ChatGPT, Claude or Perplexity.
So I didn’t settle for the statement. I went to the logs.
What the logs show: almost no bot fetches llms.txt
The number first: in the logs of dozens of large sites and SaaS products I have access to, requests for llms.txt come to fewer than one request in every ten thousand. And that’s before looking at who is asking.
The team I managed at Wix tracked the logs at the scale of hundreds of millions of sites. Actual usage: zero.
And both sets tell the same story: what little does come in, comes from unknown, obscure players. Not GPTBot, not OAI-SearchBot, not ClaudeBot and not PerplexityBot, the crawlers that feed the answers in ChatGPT and the rest.
That doesn’t prove no bot will ever fetch it. It does show what happens today: in every log set I checked, the bots that actually matter for GEO don’t bother fetching this file.
But site X added llms.txt
That’s the most common response to these numbers, and it usually arrives with a screenshot of a large, familiar site with the file sitting in its root.
Good for site X, but the fact that they added a file doesn’t mean anyone is looking at it besides other SEOs. Adding it is a decision made on one side. Usage is what happens on the other side, and as we just saw, it’s measured in logs, not in screenshots.
On the publishing side, adoption came almost entirely from documentation sites and development environments, the places where a model needs quick access to docs and APIs. That’s the audience the proposal was written for, and there the file genuinely is common.
When do you actually need llms.txt?
There’s an edge case that does make sense: AI agents navigating your site. A file that hands them a quick map can save them crawling time. Chrome notes this too, and also notes that the file is optional and that the audit is marked N/A if it doesn’t exist. And if you run a documentation site that developers point coding agents at, that’s exactly your case, and there the file genuinely belongs. For everyone else, even that use isn’t here yet.
Here’s how that looks on my own site: I run Lighthouse, I have no llms.txt, the check is marked Not Applicable, and the category still scores 2 out of 2. The check sits there next to the WebMCP checks, a protocol meant to let agents operate sites, still behind an origin trial you have to register for.
It’s prep work for an air conditioner you haven’t bought. You cut a hole in the wall for a unit that may or may not arrive. Maybe you’ll need it. Maybe not. In the meantime there’s a hole in your wall.
In the very, very best case, llms.txt is a nice to have. Nothing more.
A few months ago I sat down with a good friend from the field and we talked about exactly this. After I laid out that there’s currently no proof the file works, not even a little, he said something I agree with: if it’s easy for me, one toggle in WordPress or a CMS, why not flip it anyway?
And he’s right. If it’s a two-minute toggle, there’s not much reason to resist. Google itself writes that the file neither harms nor helps your visibility in Google Search. And it shows the people above you that you’re doing something on the AI front. The problem only starts when you prioritize it over development and improvements already waiting in the queue, or when someone charges you for it like a GEO service.
What should you do for AI visibility instead?
Instead of investing in a file the bots don’t read, focus on what actually works:
- Make sure the information about your brand and product is correct and consistent everywhere AI engines learn from.
- Make sure AI bots can crawl your site without friction.
- Build content that answers real questions, in text that lives in the visible HTML. Because that’s what the engines actually read.
- Track what ChatGPT actually searches and cites in your niche, with my free Chrome extension.
AI visibility starts with technical SEO and a consistent entity
Real AI visibility starts with solid technical SEO and a consistent entity? What is Entity? An entity is a brand, person or concept search engines and AI models treat as one thing, with a name, attributes and links to other entities. Full definition of Entity that engines like ChatGPT, Gemini and Perplexity recognize and trust. That’s exactly where technical SEO meets GEO.
Bottom line
You don’t need an llms.txt file.
Not for Google, and, based on what I actually see in the logs, not for the AI engines either. Maybe one day, as the agentic world matures, this file will serve AI agents.
Right now, you can skip it without losing anything.
I’m not one of the llms.txt opponents. I wish I had proof that it helps with something, I’d be running tomorrow to ask every one of my clients to implement it. If you have empirical evidence that the file is actually being used, send me a message or get in touch.
