Blog · Guide · Updated 20 October 2026

llms.txt: What It Is, Whether It Works, and How to Create One

By Dimitris Drakatos, Founder, Wellknown

llms.txt is a plain Markdown file at your site's root that gives AI language models a curated map of your best pages. It describes, it does not control: robots.txt holds the rules for AI crawlers, and llms.txt is the guidebook for visitors you have already let in.

What is llms.txt?

llms.txt is a proposed standard (llmstxt.org), put forward by Jeremy Howard of Answer.AI, for a Markdown file served at /llms.txt, written for an LLM reader rather than a crawler. Where a sitemap lists every URL for machines that index, llms.txt curates the fifteen to forty pages that matter for a machine that answers, each with a description that adds context a title cannot. The format is deliberately simple: an H1 with the site name, a blockquote summary (the single most valuable line in the file, because it is the sentence a model can lift to answer "what is this company?"), optional notes, then H2 sections of links grouped by intent, with an Optional section for material a model can skip when context is tight.

llms.txt vs robots.txt

The confusion this topic keeps generating, settled in two sentences. robots.txt sets crawl rules: it tells crawlers where they may and may not go, and each AI provider's automated crawlers check it for their own names. llms.txt is curated description: it tells models that already have access which pages matter and why. A disallow rule in llms.txt does nothing, and a page description in robots.txt does nothing. They are neighbours at your site root doing unrelated jobs, and a complete setup uses both deliberately. Neither file can lock anyone out, though: a fetch a person triggers, such as ChatGPT-User's, may not follow robots.txt, and only your server or CDN can turn a visitor away.

Does llms.txt work?

The short answer, which most coverage avoids giving: llms.txt is a proposed standard with growing adoption and no confirmed ranking or citation weight from any major engine. Google has not announced support. OpenAI and Anthropic have not published commitments to it. What is documented is adoption on the supply side, thousands of sites now serve one, including major documentation platforms, and the logic on the demand side is straightforward: AI systems are context-limited, curated maps are cheap to read, and the file exists at a predictable location.

Our position, stated plainly: we ship it, and we shipped ours before writing this guide. The cost is minutes, the downside is none, and the option value is real, because if and when engines lean on it, the sites already serving accurate files inherit the advantage, and if they never do, you have lost a coffee break. Treat it as one lever among several in LLM SEO, not a magic file: it curates your content for machines that can already read you, which means access and rendering still come first.

How to create an llms.txt file

The format, step by step, using our own live file, getwellknown.ai/llms.txt, as the worked example.

1. The H1: your site's name. One line, the only strictly required element. Ours: # Wellknown.

2. The blockquote summary. One or two sentences a model could quote verbatim to answer "what is this site?". Write it like the elevator pitch it is. Ours names the product, the mechanism (crawling as each real AI agent) and the audience in a single sentence.

3. Optional notes. A short prose block for anything a reader needs before the links, pricing, contact, a key caveat. No headings allowed here.

4. H2 sections of links, grouped by intent. Not by URL structure: by what someone would ask about. Ours groups Product, Blog and Optional. Each link takes the form - [Page name](url): what this page is useful for, and the description after the colon should add information, not echo the title, because that description is what tells a model whether the page answers the question in hand.

5. The Optional section. That exact heading, reserved by the spec for material a model can skip when context is short. Demote secondary pages here rather than deleting them.

Fifteen to forty links is the sensible range. A three-hundred-line llms.txt has missed the point, which is curation. A sitemap already exists for completeness.

llms.txt by platform

Shopify

Shopify now serves one for you: /llms.txt on a Shopify store returns the same content as its default /agents.md file, according to Shopify's developer changelog of 28 May 2026. To write your own, add a templates/llms.txt.liquid file to your theme under Online Store > Themes > Edit code, and it replaces the default at that address.

WordPress

Several SEO plugins now generate one, and Yoast has added llms.txt support. Alternatively, upload a hand-written file to the site root via your host's file manager. Hand-curated beats auto-generated here, because the value is editorial.

Wix

Wix generates an llms.txt automatically for sites on its platform. Review what it produced rather than assuming, since auto-generated descriptions tend to echo titles.

Webflow

Webflow supports it natively: upload the file under Site settings > SEO > LLMs.txt and publish. It is served at /llms.txt on your custom domain but not on the webflow.io staging address, and Webflow's instructions ask for a UTF-8 file under 100 KB.

Framer

On the Pro and Enterprise plans, upload the file under Site Settings > Hosting > Files and publish, and Framer serves it at /llms.txt (Framer's guide). On other plans, the fallback is a redirect from /llms.txt to a file hosted elsewhere, or a proxy layer that serves it at the root path.

Cloudflare

If your site sits behind Cloudflare, a static file in your project or a tiny Worker route serves /llms.txt cleanly. One caution that belongs in the same breath: Cloudflare's own defaults can block the very agents that would read the file, so check your AI crawler policy while you are in the dashboard.

How to validate your llms.txt

Until the dedicated tools mature, validation is a five-point manual check:

1. One H1, present and first.
2. A blockquote summary directly after it.
3. No headings other than H1 and H2.
4. Every link in the - [name](url): description shape.
5. The Optional section, if present, spelled and cased exactly ## Optional.

Then the check that matters most: open the file and ask whether a stranger, human or machine, could understand what your site is and where its value lives from this file alone. We are building a free generator and validator to automate this. Until it ships, the manual pass takes five minutes. And if you would rather have it checked properly: the Wellknown audit reviews your llms.txt as one of dozens of checks, alongside the layers that decide whether it matters at all, what each AI agent is served, what survives rendering, and whether your best pages are extractable once an agent arrives.

Frequently asked questions