LLMS.txt: Dead or Never Existed?

When llms.txt was introduced in September 2024, a certain corner of the internet got genuinely excited. Finally, a way to tell AI systems exactly how to read your website. Clean, structured, machine-readable. Elegant, even. But here we are in 2026, and the honest question is: did this thing ever really take off, or were we all just excited about a text file that nobody important actually reads?

What llms.txt Actually Is

llms.txt is a proposed web standard - a plain text file you place at the root of your website (like yoursite.com/llms.txt) that is designed to help large language models efficiently understand and access your content. Think of it as a structured guide for AI crawlers: here is what my site is about, here is what matters, here is how to read it. The idea is that instead of an LLM having to parse through messy HTML, JavaScript-heavy pages, and navigation clutter, it gets a clean, organised summary it can actually use.

It was proposed by Jeremy Howard in September 2024 at Answer.AI and drew immediate comparisons to robots.txt, which tells search engine crawlers what they can and cannot access. The difference is that llms.txt is not about access control - it is about comprehension. It is meant to make your content more legible to AI systems, which in theory should improve how well those systems represent your brand when answering user questions. If you care about AI visibility, the pitch made a lot of sense on paper.

Who Actually Adopted It

Early adoption was real, but it was concentrated in a very specific world: developer infrastructure. Companies like Anthropic, Cloudflare, Vercel, Stripe, and Hugging Face were among the first to implement llms.txt. These are organisations whose audiences are developers who actively use AI tools, so the motivation was clear. They wanted their documentation to be accurately represented when developers asked ChatGPT or Claude how to use their APIs.

By mid-2025, community-maintained directories counted over 784 websites with llms.txt implementations. That sounds decent until you zoom out. A study by Rankability looked at the top 1,000 most-visited websites globally and found that only 0.3% had implemented llms.txt as of June 2025. That is three websites out of a thousand. The tech and software sector showed stronger numbers - one industry report analyzing 2,147 websites found that 95% of tech and software companies had some form of explicit llms.txt policy - but that is a very narrow slice of the web.

So the picture is: enthusiastic early adoption among developers, near-zero adoption everywhere else. That gap tells you a lot.

The Core Problem: Nobody Is Actually Using It

Here is where things get uncomfortable. A file that tells AI systems how to read your content is only useful if AI systems actually read that file. And right now, there is very little evidence that the major players do.

"llms.txt is a proposed standard... but unless the major LLM providers agree to use it, it's pretty meaningless."

That quote cuts right to it. Google, OpenAI, and Anthropic have not publicly committed to supporting llms.txt as a standard they actively use when crawling or training. John Mueller from Google's Search Relations team compared llms.txt to the old <meta name="keywords"> tag - a relic that webmasters used to obsess over but that search engines quietly ignored. He stated that "none of the AI services have said they're using LLMs.TXT." That is a pretty damning assessment from someone who works at one of the most influential companies in search.

Understanding where ChatGPT, Perplexity, and Claude actually fetch their sources from makes this even clearer. These systems rely on their training data, real-time web crawling, and retrieval-augmented generation pipelines - none of which have announced llms.txt as an input signal. If the file is not being read, it is not doing anything.

The Google Contradiction

Here is the part I find genuinely funny. Google's official position, as stated by John Mueller, is essentially that llms.txt is not something they use or endorse. Fair enough. Except that several of Google's own developer documentation teams went ahead and implemented it anyway. At least five internal Google teams adopted the standard despite the company's public skepticism.

What does that tell us? Probably that individual teams within large organisations make pragmatic decisions regardless of top-level policy. If you are a developer documentation team and llms.txt might help your content get represented accurately in AI answers, why not add it? It costs almost nothing. But it also reveals that even Google internally is not sure what to do with this standard. That ambiguity is not a great sign for widespread adoption.

Does It Actually Do Anything for AI Visibility?

Honestly, the evidence is mixed at best. On the positive side, Vercel claimed that 10% of their signups came from ChatGPT, which they attributed partly to Generative Engine Optimization efforts that included llms.txt. That is a real number and worth noting. But it is also impossible to isolate llms.txt as the cause - Vercel is a well-known brand in developer circles, their content is high quality, and they were doing multiple GEO-related things simultaneously.

The broader research does not paint a rosy picture. After examining roughly 300,000 domains, no relationship was found between having llms.txt and how often a domain is cited in major LLM answers. It might signal intent, but it does not guarantee action.

What actually influences how AI search systems rank and surface content comes down to factors like content quality, authority signals, structured data, and how well your pages are understood by crawlers - not a text file that may or may not be read. For more on getting cited, see what schema markup can do for AI visibility or explore how to optimize for generative search engines. If you are trying to get cited in ChatGPT or Perplexity, your energy is probably better spent elsewhere.

Is llms.txt Dead?

I would say it is not dead so much as it never fully lived. It exists in a strange limbo: technically present, occasionally implemented, but not confirmed as an active signal by any major AI platform. The standard has a small but real community around it, and in specific contexts - particularly developer documentation - it makes sense to implement it because the cost is low and the potential upside exists.

But for the average website trying to improve its presence in AI-generated answers? llms.txt is probably not where you should be spending time. The core ranking factors for AI search are about content quality, trustworthiness, and how well AI systems can parse what you actually say. A text file pointing to your content does not substitute for the content itself being excellent.

There is also a broader lesson here about how web standards actually succeed. robots.txt worked because search engines agreed to respect it and built it into their crawlers from early on. sitemap.xml worked for the same reason. llms.txt has not cleared that hurdle. Without a formal commitment from OpenAI, Google, or Anthropic to actively use it, it remains a community proposal rather than a functioning standard. That might change. But as of now, it has not.

If you are thinking about what does not actually influence AI visibility, llms.txt is a reasonable candidate for that list - at least until the major platforms say otherwise. For more strategic guidance, check our main Lumentir AI Search Hub.