llms.txt is a proposed convention for a Markdown file at the root of your site (yoursite.com/llms.txt) that gives large language models and AI agents a short, curated overview of the site with links to the most useful pages. It is not an official standard, adoption by AI search engines is uneven, and it does not replace crawl access or good content.
Where llms.txt came from
The format was proposed in September 2024 by Jeremy Howard of Answer.AI and is documented at llmstxt.org. The idea: websites are built for humans, full of navigation, scripts and layout, and AI tools working within a limited context window benefit from a clean, prioritized summary in Markdown.
What the file looks like
The proposal defines a simple structure:
- An H1 with the site or project name (the only required element).
- A blockquote with a one or two sentence summary.
- Optional paragraphs with extra context.
- H2 sections containing lists of links, each with a short description.
- An optional “Optional” section for secondary links that can be skipped.
A short example for a B2B SaaS company:
# Acme Security
> Acme is an API security platform for
> fintech and SaaS engineering teams.
## Product
- [Platform overview](https://acme.com/platform): what it does
- [Pricing](https://acme.com/pricing): plans and limits
- [Integrations](https://acme.com/integrations): CI/CD, SIEM
## Docs
- [Quickstart](https://docs.acme.com/quickstart)
- [API reference](https://docs.acme.com/api)Some sites also publish llms-full.txt, which contains the full text of key docs in one file.
Do AI engines actually use it?
This is the honest part. As of late 2026:
| Consumer | Status |
|---|---|
| Google AI Overviews and AI Mode | Google says no special files are needed; Google representatives have said Search does not use llms.txt |
| ChatGPT, Perplexity, Copilot, Claude (search features) | No public commitment to use llms.txt for choosing or ranking sources; they rely on their own crawlers and indexes |
| AI coding assistants and agents | Some tools and docs platforms read or generate llms.txt, and developers often paste it into tools manually |
| Documentation platforms | Several docs hosting platforms generate llms.txt automatically |
In other words, llms.txt is most useful where an AI tool or a person deliberately fetches it, for example a developer pointing a coding assistant at your docs. There is no public evidence that adding the file, on its own, raises your citations in AI search answers.
So, do you need one?
It depends on what you sell and what is already in place.
Add one (low effort, reasonable value) if:
- You have developer documentation, an API or an SDK.
- Your buyers are technical and use AI coding tools.
- Your docs platform can generate it automatically.
Treat it as optional if:
- You are a marketing-led site without developer docs.
- You still have crawl, rendering or content gaps. Fix those first; they matter far more.
Do not expect it to:
- Unblock AI crawlers you have blocked in robots.txt or at your CDN.
- Make a JavaScript-only page readable to crawlers that do not render it.
- Replace answer-first content or third-party authority.
What matters more than llms.txt
If your goal is AI search visibility, prioritize:
- Crawler access. Allow the search crawlers of engines you want to appear in, such as OAI-SearchBot for ChatGPT search (OpenAI bots) and PerplexityBot. Check CDN and bot-protection rules, which often block them silently.
- Server-rendered HTML. Key facts should be in the initial HTML, not loaded by client-side JavaScript.
- Answer-first content for the questions buyers ask. See what AEO is.
- Third-party corroboration. See how AI engines choose which sources to cite.
If you add one: a checklist
- Place it at the root: /llms.txt, served as plain text or Markdown.
- Start with an H1 name and a one-sentence summary that states your category clearly.
- Link to 10 to 30 of your most useful pages, each with a short description.
- Use canonical URLs that return 200, not redirects.
- Prefer pages with clean, server-rendered content (or link to Markdown versions).
- Generate it from your CMS or docs build so it stays current.
- Keep pricing, product names and positioning consistent with the rest of the site.
How to explain it to stakeholders
If a CMO or client asks whether you have done “the llms.txt thing,” a fair answer is:
- It is a voluntary, proposed format, not a standard any major search engine has committed to.
- It costs little to add and helps AI tools that do read it, especially for technical documentation.
- It is not a ranking or citation lever on its own, so it should not appear as a headline result in reports.
Framing it this way avoids overpromising and keeps attention on the work that moves AI visibility.
llms.txt vs robots.txt
robots.txt controls access and is widely respected by major crawlers. llms.txt is a suggestion about what to read. They solve different problems, and you need robots.txt configured correctly regardless. The blog post on llms.txt vs robots.txt compares them in detail, and the complete llms.txt guide covers generation and examples.
Bottom line
Add an llms.txt if it is cheap for you, especially if you have developer docs. Do not report it as a GEO win or expect it to move citation rate. Spend the real effort on crawl access, content clarity and authority.
Frequently asked questions
Does Google use llms.txt?
Google's documentation says no special files or markup are needed to appear in AI Overviews or AI Mode, and Google representatives have said publicly that Google Search does not use llms.txt. Normal indexing and snippet eligibility are what matter.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may and may not fetch, and crawlers widely respect it. llms.txt is a voluntary content map that suggests what is most useful to read. It grants or blocks nothing.
What is llms-full.txt?
An optional companion file that contains the full text of your key documentation in one Markdown file, so an AI tool can load it in a single request. It is most common on developer documentation sites.
Can llms.txt hurt my site?
Not directly. The main risks are wasted effort and a stale file that points to outdated pages. Keep it short and generate it automatically if possible.
