What llms.txt actually is
llms.txt is a plain-text (Markdown) file served at /llms.txt that gives AI systems a curated summary of your site: what it is, and which pages matter most, with short descriptions. The convention was proposed by Jeremy Howard (Answer.AI) in September 2024 as a counterpart to robots.txt — robots.txt tells crawlers what they may read, llms.txt tells language models what they should read and how to understand it.
The format is deliberately simple: an H1 with your site or product name, a one-paragraph blockquote summary, then sections of links with one-line descriptions. Because it is short, clean Markdown at a predictable URL, an AI system (or an agent browsing on a user's behalf) can ingest it in a single request instead of crawling dozens of templated HTML pages full of navigation and cookie banners.
Honest status in 2026: promising, not guaranteed
Be wary of anyone selling llms.txt as a magic ranking lever. Major AI companies have not formally committed to consuming it the way search engines committed to sitemaps, and public evidence of direct ranking impact is thin. What is true: adoption has spread across developer-tool and SaaS companies, the file is fetched by a growing set of agents and AI dev tools, and it costs almost nothing to maintain.
The sensible frame is asymmetric bet: five minutes of work, zero downside, plausible upside as agentic browsing grows. It also doubles as a genuinely useful artifact — a canonical, self-audited summary of what your most important pages are. Many teams discover while writing it that their own 'money pages' are hard to describe, which is a content problem worth knowing about.
Writing a good llms.txt
Lead with clarity, not marketing. The blockquote summary should say what you do, for whom, in plain declarative language — the same register you would use explaining your business to a competent stranger. Every link should answer a question an AI might need: what does this product cost (pricing page), how does it work (docs or methodology), where is the proof (case studies), what changed recently (changelog).
Keep it under a few hundred lines and update it when your site structure changes. Do not list every blog post; list the pages you would want quoted. If you maintain both llms.txt and a fuller llms-full.txt, the short file should stay genuinely short. And make sure the pages you link are themselves crawlable — an llms.txt pointing at bot-blocked pages is self-defeating (check with our AI Crawler Checker).
Beyond the file: being legible to machines
llms.txt is one piece of a broader discipline sometimes called GEO (generative engine optimization): structuring your site so machines can understand and confidently cite it. The rest of the checklist: server-rendered HTML for key pages, JSON-LD structured data (Organization, Product, FAQPage), consistent business name/address/phone across the web, real third-party citations (directories, reviews, press), and content that directly answers the questions buyers ask assistants.
Measurement closes the loop. After you ship llms.txt and content fixes, the question is whether AI answers actually change. That is what RankedByAI's free scan gives you: a baseline of how often AI recommends you today, and weekly monitoring to see whether your changes move it.