llms.txt is a proposed Markdown file placed at a website's root that gives large language models a short summary of the site and a curated list of links to its most useful content.
What is llms.txt?
llms.txt is a proposed convention for a plain Markdown file at /llms.txt that tells large language models what a website is about and where its most important content lives. It was proposed by Jeremy Howard of Answer.AI in 2024 and is documented at llmstxt.org. It is a guide for models, not an access control file.
How it works
The proposal describes a simple structure:
- An H1 with the site or project name.
- A short blockquote summary.
- Optional paragraphs of context.
- H2 sections with Markdown lists of links, each with a brief description.
Some sites also publish /llms-full.txt, a single file with the full text of key docs, so an AI tool can load it in one request. The format is popular with developer documentation because coding assistants and agents can fetch it directly.
What it does not do matters as much. It does not control crawling (that is robots.txt), it is not a ranking signal any search engine has announced, and major AI search providers have not publicly committed to reading it. Its clearest value today is for agents and tools that a user points at your site.
Examples
- A developer platform publishes llms.txt linking to its quickstart, API reference and SDK pages, so coding assistants find the right docs.
- A B2B SaaS company lists its product overview, pricing, security page and comparison pages with one-line descriptions.
- A documentation site adds llms-full.txt so an agent can load the full reference without crawling dozens of pages.
How to apply it
- Write a one-sentence description of what the company or product does.
- Pick 10 to 30 pages that answer the most common questions about you.
- Group them under clear H2s and add a short note to each link.
- Keep facts consistent with your main pages and update when they change.
- Do not rely on it in place of crawlable HTML, structured data and good content.
Related terms
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal published at llmstxt.org in 2024. Support by major AI search engines has not been publicly confirmed, so treat it as low-cost and optional.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may access. llms.txt does not grant or block access; it points models to the content you consider most useful.
What should go in llms.txt?
A one-line summary of the site, a short description, and grouped links to key pages such as product overview, documentation, pricing and core guides, ideally with a short note per link.
Guides that cover llms.txt
-
LLMs.txt vs Robots.txtLLMs.txt vs Robots.txt: How to Optimize Your Website for AI Crawlers & Answer Engines
LLMs.txt vs Robots.txt explained: what each file does, who reads it, and how to implement both to optimize your site for AI crawlers and answer engines
-
AI citationsHow to Track AI Citations, Mentions & Sources Across ChatGPT, Claude, Gemini & Perplexity
Learn how to track AI citations, mentions, and sources across ChatGPT, Claude, Gemini, and Perplexity with a repeatable, per-engine workflow
-
Claude SEOClaude SEO Guide: How to Track, Measure & Improve Your Brand Visibility in Claude AI
Claude SEO guide: track, measure, and improve your brand visibility in Claude AI step by step, with concrete tactics for citations, prompts, and content.
-
LLMs.txt: The Complete Guide to Making Your Site AI-Readable
LLMs.txt: The Complete Guide to Making Your Site AI-Readable
