Alien Road uses llms.txt as a small, low-cost part of generative engine optimisation work: a curated Markdown file at the site root that points AI tools to a client’s most important pages with short factual notes. We treat it as hygiene, not a ranking tactic. The real work in a campaign happens in the content and technical SEO that AI answers draw on; llms.txt simply gives agents a clean map of that content.
The llms.txt format was proposed in 2024 by Jeremy Howard of Answer.AI. It defines a simple structure: a title, a short summary, and sections of links with descriptions. At the time of writing, it is not an official standard and the major AI search products have not confirmed using it for ranking or citation. Several documentation platforms and developer tools do read it, which is why it is worth an hour of work but not more.
Why include llms.txt at all if its impact is uncertain?
Three reasons. It is cheap: a well-made file takes little time. It forces useful decisions: choosing the twenty pages that best describe a business is a valuable exercise in itself. And it costs nothing in risk, because it does not change how search engines crawl or index the site.
What we do not do is promise results from it. Clients hear clearly that citations in AI Overviews, ChatGPT or Perplexity depend on indexability, content clarity, authority and structured data, not on this file.
Step 1: Decide which pages belong in the file
The file is a curated list, so selection matters. We start from the questions buyers ask AI assistants about the client’s category and find the pages that answer them best.
- Core service or product pages that describe what the business does
- Pricing, packages or how-to-buy information
- About page with location, history and team
- Policies a buyer checks: shipping, returns, privacy, terms
- Documentation or guides for technical products
- Two or three cornerstone articles that answer high-value questions
Blog archives, tag pages, thin landing pages and campaign microsites stay out. If a page would not help a new employee explain the business, it does not belong.
Step 2: Write the file in plain, factual language
The summary at the top states what the business is, where it operates and who it serves in two or three sentences. Each link gets one line explaining what the page answers. We avoid adjectives and slogans, because language models extract facts and skip praise.
For example, a line might read: Pricing: monthly packages for SEO and paid media, with what each includes and minimum term. That is far more useful to a model than a line calling the page our amazing offers.
Step 3: Validate every link and the server setup
- Every URL returns 200 with no redirect chains
- Each page is canonical and indexable
- robots.txt allows the AI user agents the client has chosen to permit
- The file is served at the root as plain text in UTF-8
- Page caching and CDN rules serve the current version
- Markdown versions of key pages, if offered, match the HTML content
We also check consistency with the client’s crawler policy. If a client has decided to block certain AI crawlers, listing pages for them sends a mixed message, so we align the two decisions.
How does llms.txt fit with the rest of a GEO campaign?
In a typical engagement the file is one item among many. The larger pieces of work usually include rewriting key pages so the direct answer appears first, adding Organization, Product or Service schema, fixing indexing issues in Search Console, building consistent entity information across listings, and earning mentions on sites AI tools often cite.
llms.txt benefits from that work rather than driving it. Once key pages are clear and well structured, the file points to good material. If those pages are weak, the file only points agents to weak material faster.
How do we measure whether the file is used?
Measurement is through server logs. We look for requests to /llms.txt and to linked Markdown pages, and we group them by user agent. Known AI crawlers and agent tools identify themselves, though not all do, and some requests are simply curious developers or SEO tools.
Reporting separates the two questions clients care about. First, is the file being fetched, and by what? Second, are AI citations for the client’s priority questions changing? The second is tracked through a fixed query set checked regularly in AI Overviews, ChatGPT search, Perplexity and Copilot. We do not attribute changes in the second to the first without evidence.
What maintenance does the file need?
Sites change. Services are renamed, pages merged and policies updated. We add the file to the post-launch checklist used for redirects and sitemaps, and run an automated monthly check that fetches every listed URL and flags anything that is not a clean 200.
Multilingual sites need a decision. Some clients prefer one file in the main language with language sections; others publish separate lists per language path. Either works if it is consistent and maintained.
What clients often ask
Will this get us into ChatGPT answers?
Not on its own. ChatGPT search and similar tools rely on search indexes and their own crawlers. The file may help agent-style tools that read it, but citations depend mainly on content and authority.
Should we publish llms-full.txt too?
For documentation-heavy products, often yes, because developer tools may load the full text. For most service businesses, the shorter file is enough.
Who owns the file inside the client team?
We ask clients to name an owner on their side, usually the person responsible for the website or SEO. That person approves the page list, receives the monthly link check and decides whether new pages should be added after launches.
Ownership matters because the file sits outside normal content workflows. Nobody edits it by accident, but nobody updates it by accident either. A named owner and a calendar reminder are usually enough to keep it accurate.
Where to start this week
Write down the ten questions your buyers most often ask before contacting you, and next to each one note the page on your site that answers it best. If any question has no good page, that gap matters far more than llms.txt. For the questions that do have pages, draft a short llms.txt with one factual line per link, validate the URLs and publish it at your root.
Keep reading: GEO – AIO – AEO · GEO