llms.txt is a proposed standard for a plain Markdown file placed at the root of a website, at /llms.txt, that gives large language models a short, curated map of the site’s most useful content. It is a convenience for AI tools that read websites, not a ranking signal or an access control. Marketers can add one quickly, but should not expect it to change visibility in Google or AI answer engines on its own.
The proposal was published by Jeremy Howard of Answer.AI in 2024. The idea is simple: web pages are full of navigation, scripts and layout that make them awkward for language models to read within limited context windows. A clean Markdown summary pointing to the important pages, ideally with Markdown versions of those pages, helps an AI tool find and understand what matters.
What does an llms.txt file contain?
The format is intentionally loose and human-readable. A typical file includes:
- An H1 heading with the site or project name.
- A short blockquote summarizing what the site or company does.
- Optional paragraphs with key context, such as who the product is for.
- H2 sections with lists of links, each with a short description, for example “Services”, “Guides”, “Pricing”, “Documentation”.
- An optional section, often titled “Optional”, for secondary links that can be skipped when context is limited.
Some sites also publish /llms-full.txt, a single file containing the full text of key pages. Documentation platforms such as Mintlify and several developer tool companies were early adopters.
How is it different from robots.txt and sitemap.xml?
These three files are often confused, but they do different jobs.
- robots.txt tells crawlers which paths they may or may not fetch. It is about permission, and crawlers such as Googlebot, GPTBot and ClaudeBot read it.
- sitemap.xml lists URLs for search engines to discover, with optional dates. It is about completeness.
- llms.txt offers a curated, readable summary for language models. It is about understanding, and it lists only what you think matters most.
Blocking or allowing AI crawlers is still done in robots.txt. llms.txt cannot stop anyone from using your content.
Does llms.txt help SEO or AI visibility?
At the time of writing, there is no public confirmation that major search engines or AI answer products use llms.txt as a ranking or citation signal. Google representatives have publicly said that Google Search does not use it. Server log analyses shared by various SEOs have shown little or no regular fetching by the big AI crawlers.
Where it does get used is in tools that fetch a site on demand: coding assistants, AI agents and developer tools that look for /llms.txt when asked to read documentation. For a software company, that use is real. For a local service business, the practical benefit is smaller.
So treat it as a low-cost, low-risk addition. It takes an afternoon, it forces you to decide which pages represent the business best, and it may help as more tools adopt it. It does not replace the work that actually drives visibility.
What actually drives visibility in AI answers?
If the goal is to appear in ChatGPT, Gemini, Perplexity, Copilot or Google AI Overviews, the same fundamentals matter more than any single file:
- Crawlable, fast pages with server-rendered text that crawlers can read.
- Clear answers near the top of pages, under headings phrased as questions.
- Structured data such as Organization, Product, FAQ and Article schema where appropriate.
- Consistent entity information: company name, address, services and founders the same everywhere.
- Mentions and links from trusted third-party sites, reviews and directories.
- robots.txt rules that allow the AI crawlers you want to be visible in.
llms.txt sits alongside these as a small extra, not instead of them.
How do you write a good llms.txt?
Choose the pages
Pick 10 to 30 pages that best explain the business: core service or product pages, pricing, about, contact, key guides and policies. Skip thin pages, tag archives and campaign landers.
Write useful descriptions
Each link gets one sentence saying what the reader will find. “SEO services: technical audits, content strategy and link building for B2B companies” is more useful than “Our SEO page”.
Keep it current
An outdated llms.txt pointing to removed pages is worse than none. Add it to the checklist whenever a key page is created, renamed or retired.
A simple structure
- Company name as the H1.
- One-sentence summary in a blockquote.
- Two or three sentences on who you serve and where.
- Sections for services, resources, company information and contact.
- An optional section for secondary resources.
Common mistakes
- Treating llms.txt as a permissions file and expecting it to block AI training.
- Listing hundreds of URLs, which defeats the purpose of a curated summary.
- Writing marketing slogans instead of factual descriptions.
- Forgetting to serve it as plain text at the exact root path.
- Claiming to clients that it guarantees AI citations.
How Alien Road applies llms.txt
In a typical generative engine optimization engagement, llms.txt is one small item in a larger technical review. Discovery starts with how AI systems currently describe the client: we ask ChatGPT, Gemini, Perplexity and Copilot a set of brand and category questions and record which sources they cite and what they get wrong.
Setup focuses on the fundamentals first: crawler access in robots.txt, structured data, consistent entity information and clear answer-first content on key pages. The llms.txt file is then written from the same list of priority pages, with factual one-line descriptions, and placed at the site root. For clients with documentation, we also consider Markdown versions of key pages.
Measurement does not attribute results to llms.txt alone, since that cannot be isolated honestly. We check server logs for requests to the file, and repeat the AI answer questions monthly to track whether brand descriptions become more accurate and citations increase. Reporting is clear about which changes are likely responsible.
Where to start this week
Write down the 15 pages you would show a new employee to explain your business. Turn that list into an llms.txt file with a one-sentence summary and a short description for each link, and publish it at your site root. Then spend the rest of the time checking robots.txt for AI crawler rules, which has a far bigger effect on AI visibility.
Keep reading: GEO – AIO – AEO · GEO