How to Implement llms.txt for AI Search Discovery

llms.txt is unusual: it is a low-effort, low-risk publication that delivers a measurable lift on a fast cycle. The cost is a single static file. The upside is a 20–40% citation lift on the two engines that ingest it most explicitly. The reason it is not yet universally implemented is that most teams treat the file as a one-time content artifact instead of a build-time output of the same source the site is built from — which is exactly the mistake this guide is built to head off.

What llms.txt Is, in One Paragraph

llms.txt is a proposed file-based standard (introduced by Jeremy Howard in September 2024) that lets a site publish a markdown-formatted, LLM-friendly summary of its content at the root path /llms.txt. The file is modeled on robots.txt and sitemap.xml: a single static file, served from the site root, ingested by user agents that honor it. The companion file /llms-full.txt embeds the full markdown content of the linked pages for direct LLM ingestion. The standard is voluntarily implemented by hundreds of high-traffic sites by mid-2026 and is honored by Perplexity, ChatGPT, Claude, and an increasing number of vertical AI assistants.

What a Good llms.txt Looks Like

The minimum viable structure:

• H1. The brand name, exactly as you want engines to render it.
• Intro paragraph. Three to four sentences: what the brand is, who it serves, the categories the site covers. The intro is the first thing the engine reads and frames every retrieval decision that follows.
• H2 sections. One H2 per topical cluster — Guides, Glossary, Comparisons, Case Studies, FAQ, Pricing. Order by retrieval priority; the engines weight earlier sections more heavily.
• Link list per section. Markdown links with one-line descriptions. Concrete (one verb, one audience, one outcome) and short (under 150 characters).

The single highest-leverage section is usually the glossary or FAQ — engines treat these as dense, factual surfaces and lean on them hard for category-defining queries. If the site has a glossary, the glossary section should be one of the top three H2s in the file.

Build-Time Generation Is the Step That Matters

The single most common reason llms.txt programs decay is that the file is hand-edited. By month three, the file references pages that have been renamed, omits new pages that should be there, and contains stale descriptions for pages that have evolved. The fix is to generate the file at build time from the same data sources that drive the site — for a Next.js site, this is a route handler at app/llms.txt/route.ts that reads the glossary, guide, comparison, and FAQ data files and emits the markdown.

The build-time pattern has three side benefits worth naming: (1) the file is automatically in sync with the rest of the site, (2) CI can validate every linked URL on every merge, and (3) the file becomes a forcing function for clean source data — when the glossary entry is missing a description, the llms.txt is the surface where the gap becomes visible.

llms.txt vs llms-full.txt: When to Ship Each

Two practical patterns:

• llms.txt only. Ship for every site over 50 canonical pages. The link-map version is the higher-ROI first investment and is what every engine honors.
• llms.txt + llms-full.txt. Add the full-content version for the top 30–80 pages once the link-map version has stabilized. Keep it under 10MB; engines retrieval-timeout above that threshold. The full version most clearly helps Perplexity and Claude, both of which ingest the full markdown when present and use it as the retrieval substrate for the cited site.

Sites under 30 pages should skip llms-full.txt entirely — there is not enough content depth to make the full-content ingest meaningful, and the maintenance overhead does not justify the marginal lift.

Where the Implementation Breaks

• Hand-maintained files. Decay inside two quarters. Generate at build time or do not ship at all.
• 404 rot. Linked URLs that 404 are treated as a quality penalty by the engines. Wire CI to fail the build on the first 404.
• Description vagueness.‘Our guide to AI UGC’ gives the engine nothing to route on. Concrete descriptions (verb + audience + outcome) materially out-retrieve generic ones inside the same file.
• Section ordering.Putting Pricing or Company at the top of the file wastes the engine’s attention weight. Top three H2s should be the topical clusters you most want cited.
• Pre-launch measurement absence. Sites that skip the two-week pre-launch baseline cannot attribute the lift cleanly and ship the wrong follow-on investments.

How llms.txt Fits the Full AI-Search Stack

llms.txt is one of four artifacts a mid-2026 GEO program runs on — the visibility dashboard, the entity graph audit, the rationale snippet audit, and the discoverability stack (llms.txt, schema, sitemap). The file is the lowest-effort artifact of the four. Brands that lead with it before the other three see the citation lift but cannot attribute it cleanly; brands that ship it after the other three see the lift compound on top of an already-disambiguated entity layer.

Related reading: the visibility tracking dashboard, the brand entity graph audit, the rationale snippet audit playbook, and the content gap audit are the four artifacts that compound with a maintained llms.txt — together they form the five-artifact AI-search measurement stack a 2026 GEO program runs on. The flip side also matters: a clean, maintained llms.txt is the surface a competitor’s footprint map will read against you, so publishing one is the defensive posture for the offensive program described in the citation footprint mapping playbook.