llms.txt: what it is and whether your site needs one
A one-page markdown map of your site, written for language models rather than people. The idea is elegant, the cost is an afternoon, and the evidence that anything reads it is thin. Here is the whole picture.
llms.txt is a proposed standard: a plain markdown file at the root of your domain that lists your most important pages with one-line descriptions, so an AI model can grasp your site without wading through its HTML. It costs about an hour to add. As of today, no major AI platform has publicly committed to reading it in production, and Google says its AI features do not use it.
In short
- llms.txt was proposed by Jeremy Howard of Answer.AI on 3 September 2024: a curated markdown index of a site, served at /llms.txt.
- The format is strict markdown: an H1 with the site name, a blockquote summary, then sections of bulleted links with one-line descriptions.
- Google does not support it. Gary Illyes said so directly, and John Mueller compared it to the old keywords meta tag.
- Honest verdict: cheap and harmless, no proven effect on rankings or AI citations. We publish one at neurise.io/llms.txt anyway.
What llms.txt actually is
llms.txt is a text file that acts as a curated table of contents for language models. Jeremy Howard, co-founder of Answer.AI, published the proposal on 3 September 2024, and a small community maintains it at llmstxt.org. His argument was practical: model context windows are limited, and real web pages bury their substance under navigation, adverts and JavaScript. A single markdown file that says "here is who we are, and here are the pages that matter" lets a model get to the point in one cheap request.
It is worth being precise about what it is not. robots.txt tells crawlers what they may fetch. An XML sitemap lists everything you have, without opinion. llms.txt is deliberately opinionated: a short, human-written selection of the ten or twenty pages you would show a new client, each with a one-line description. The proposal also describes an optional companion, llms-full.txt, which flattens the full content of those pages into one large markdown document for models that want everything at once.
The format, with an example
The specification is short. One H1 with the site or project name (the only required element), an optional blockquote with a one-sentence summary, then H2 sections containing bulleted links in the form [name](url): description. An "Optional" section at the end marks links a model can skip when context is tight. A trimmed version of our own file looks like this:
# Neurise > AI-native SEO and GEO agency. We make brands visible in > Google and in AI answers (ChatGPT, Gemini, Perplexity). ## Key pages - [GEO services](https://neurise.io/en/geo/): how we optimise brands for generative engines - [Free SEO and GEO audit](https://neurise.io/en/seo-audit/): what we check and what you receive within 5 working days ## Optional - [Case studies](https://neurise.io/en/case-studies/): client results with numbers
The full version sits at neurise.io/llms.txt. Note what the format forces you to do: pick your most important pages and describe each one in a single factual line. That editorial exercise has value even if no model ever fetches the file.
Who supports it, and who does not
Adoption split early into two camps: publishing the file and reading it.
Publishing took off in developer documentation. Mintlify, which hosts docs for many software companies, rolled out llms.txt for all of its hosted sites in November 2024, which put the file on thousands of docs domains almost overnight, Anthropic's among them. Anthropic worked with Mintlify on llms.txt and llms-full.txt for its own documentation, and OpenAI serves one on its developer docs site too. For technical documentation aimed at coding assistants, the file has become a fairly normal courtesy.
Reading is another matter. According to reporting by PPC Land, no major AI platform (OpenAI, Google, Anthropic or Meta included) has publicly committed to reading llms.txt in its production systems, and the server-log analyses that publishers have shared report little crawler interest in the file. Publishing one and consuming one are different decisions, and so far only the first has happened at scale.
Google has been openly sceptical. Gary Illyes said at a Search Central Live event that Google does not support llms.txt and has no plans to. John Mueller compared it to the old keywords meta tag: a self-declared claim about your own content that a system would have to verify against the real pages anyway, at which point it may as well just read the pages. Google has also stated that sites do not need the file to appear in its AI features: AI Overviews and AI Mode draw on ordinary crawling and indexing, the same infrastructure explained in our guide to Google AI Mode.
Publish llms.txt the way you hand out a business card: cheap, tidy, and with no illusion that it closes the deal.
The honest verdict
Our position, as an agency that sells GEO (Generative Engine Optimization): llms.txt is worth an hour, and not worth a strategy. It cannot hurt you, it costs almost nothing, and if agent-style tools begin fetching it routinely, you are already there. What it will not do, on current evidence, is move rankings or AI citations, because the systems that produce those answers are not reading it.
The mechanism that actually decides whether you appear in AI answers is retrieval. Assistants split a question into sub-queries through query fan-out, pull back passages from indexed pages, and cite the sources those passages came from, a process we unpack in where AI citations come from. Your leverage sits in three places: content written in liftable, answer-first sections, explicit structured data, and clean crawler access, which means getting your robots.txt right for the AI crawlers that matter. An llms.txt file behind a robots.txt that blocks GPTBot is a menu posted on a locked door.
How to add one in an afternoon
- Choose 10 to 20 pages you would show a new client: core services, pricing, proof, your best explainers. Leave out tag archives and thin pages.
- Write the markdown: H1 with your brand name, a blockquote of one or two factual sentences, then sections of links with one-line descriptions. No adjectives a model would have to verify.
- Upload it to your domain root as /llms.txt, served as plain text, and check it loads with a simple request. Add llms-full.txt only if you have clean markdown versions of the pages to concatenate.
- Revisit it when the site changes, ideally in the same checklist as your sitemap. A stale curated index is worse than none, because it misdescribes you with confidence.
If you want the file reviewed alongside the things that do carry weight (retrievability, schema, crawler access), that is part of our free SEO and GEO audit.
Common questions
What is llms.txt?
A proposed standard: a markdown file at the root of your domain that lists your most important pages with one-line descriptions, so an AI model can grasp the essentials of your site without parsing your full HTML.
Does Google use llms.txt?
No. Google's Gary Illyes has said Google does not support the file, and John Mueller has compared it to the old keywords meta tag. Google's AI features rely on normal crawling and indexing instead.
Will llms.txt improve my AI visibility?
There is no public evidence yet that it does. Treat it as a cheap, harmless addition and spend your real effort on retrievable content, structured data and crawler access.
How do I create an llms.txt file?
Write a markdown document with an H1 for your site name, a short blockquote summary and bulleted links to key pages with one-line descriptions, then upload it to your domain root as /llms.txt. Ours at neurise.io/llms.txt takes exactly that shape.
Read next
Find out whether AI recommends your company.
Start with the free SEO and GEO audit, delivered in 5 working days. We check how the models describe your brand and hand back a prioritised list of changes.