How to Implement llms.txt: Format, Steps and Honest Limits

Treat llms.txt as a small piece of infrastructure: a curated, plain map of your site for tools that look for one, mainly coding assistants, documentation tools and agent frameworks. This guide covers the format as the spec defines it, the related files people mean when they say “llms.txt”, the step-by-step setup, and what has actually been said about whether AI search engines read it.

What llms.txt Is (and Isn't)

llms.txt is a proposal for a markdown file at the root of a site that gives a language model a short, clean orientation: what the site is, and which pages hold the useful content. It sits next to robots.txt and sitemap.xml but does a different job. robots.txt says what a crawler may fetch; a sitemap lists what exists; llms.txt says what matters and what each page is for.

It is not:

• An access control. Crawler permissions still live in robots.txt. OpenAI, for example, documents separate user agents for search (OAI-SearchBot), model training (GPTBot) and user-triggered fetches (ChatGPT-User), each controlled in robots.txt.
• A confirmed ranking or citation signal. Google's AI features documentation says: “You don't need to create new machine readable files, AI text files, or markup to appear in these features.” Google's John Mueller has compared llms.txt to the keywords meta tag (Search Engine Journal). We know of no public statement from OpenAI, Perplexity, Microsoft or Anthropic that their search products use it to choose sources.
• A substitute for good pages. AI search systems fetch and read your normal HTML. If a page doesn't state its answer plainly, pointing to it from llms.txt changes nothing. See how AI search retrieves and cites passages.

Where it does get used: documentation platforms and SEO plugins generate it (llmstxt.org lists Mintlify, GitBook, Yoast SEO and others), and some developer tools fetch it as a fast path to clean docs. OpenAI's own API docs publish one and serve markdown versions of pages when you append .md. If developers or agents use your product, that is the concrete payoff.

The Format, per the Spec

The llmstxt.org spec defines the sections in this order:

1. H1 with the site or project name. The only required part.
2. A blockquote with a short summary of the key facts needed to understand the rest of the file.
3. Optional detail sections: paragraphs or lists (any markdown except headings) with more context.
4. H2 sections of “file lists”: markdown lists where each item is a link, optionally followed by a colon and a note.
5. An H2 named “Optional”, by convention, for links that can be skipped when a shorter context is needed.

A minimal example:

# Acme Lamps

> Acme Lamps sells dimmable LED desk lamps direct to consumers in the US.
> Prices start at $49. We do not sell replacement bulbs.

## Product
- [Lamp range](https://acme.example/lamps.md): every model with specs and prices
- [How dimming works](https://acme.example/dimming.md): modes, color temperature, power use

## Policies
- [Shipping and returns](https://acme.example/returns.md): 30-day returns, US-only shipping

## Optional
- [Company history](https://acme.example/about.md)

The Related Files

“llms.txt” is often used as shorthand for a small family of files. Only the first and third are in the llmstxt.org proposal.

Start with llms.txt and .md versions of the few pages where your facts live. Add llms-full.txt only if your audience is developers who paste docs into a coding assistant; for most marketing sites it is a large file that has to be kept in sync for little gain.

Writing a File That Earns Its Place

• Curate, don't dump. A copy of your sitemap tells a reader nothing new. Pick the pages that answer the questions people ask about you.
• Lead with facts models get wrong. If assistants often misstate your price model, your market or what you don't do, say it in the blockquote in plain sentences. The same facts should be on your About and pricing pages; see our brand entity guide for fixing wrong facts at their source.
• Group by question, not by menu. “Pricing and policies” and “How it works” are more useful headings than your top-nav labels.
• Write concrete notes. One line: what the page answers and for whom. “Our guide to AI UGC” is a wasted note.
• Keep it short. If a reader has to skim a thousand links, the file has stopped being a map.

For a working example, see ppl.studio's own llms.txt. It is hand-curated on purpose: a short, opinionated map with a machine-readable /pricing.md, reviewed whenever plans change. Build-time generation suits large docs sites; a short curated file suits a product site. Either way, the file must not drift from the pages it describes.

Agent-Readable Pages Matter More Than the File

Whatever reads llms.txt then fetches your pages, and so does every AI search crawler that ignores it. The checks that help all of them:

• Content in the HTML. Key text should be server-rendered, not injected after load, and not hidden behind tabs that need a click to fetch.
• Crawler access set on purpose. Decide per user agent in robots.txt (search vs. training vs. user-triggered fetches) rather than blocking or allowing every AI bot by accident. Our OAI-SearchBot and GPTBot entries explain the difference.
• Structured data that matches the page. Organization, Product, FAQPage and Article markup describe the page in a form any parser can read. Google notes there is no special schema needed for its AI features, but markup that matches visible content is still how you state facts unambiguously. See schema markup.
• Current facts. Prices, availability and policies change. Tie llms.txt and the .md pages to the same review as the pages themselves; our content refresh calendar guide covers the cadence.
• Product data in a feed, not just prose. Shopping assistants work from product feeds and structured product data; llms.txt carries the narrative and policy context around them. The AI shopping assistants playbook covers the feed side.

Common Mistakes

• Expecting a traffic or citation lift. Nothing public supports it. If citations rise after you ship the file, check what else changed.
• Stale facts. An old price in llms.txt misinforms the one reader the file was written for.
• Dead links. Renamed pages that now 404 or redirect. Validate on every deploy.
• Blocking the file. A robots.txt rule or bot protection that serves a challenge page instead of the file.
• Claims the pages don't back up. The summary should repeat what your pages say, not add marketing claims that appear nowhere else.

Measuring It Honestly

Two separate questions. First, does anything fetch the file? Your server or CDN logs answer that: filter requests for /llms.txt and .md pages and group by user agent. Second, are you cited more in AI answers? That is measured by your regular AI-visibility tracking and AI referral traffic, described in our AI search attribution guide, and it is driven by your pages, not by this file. Keep the two apart and you won't credit a cheap file with results it didn't cause.