Blog · Guide

llms.txt: What It Is, Whether It Works, and How to Create One

By Dimitris Drakatos, Founder, Wellknown

llms.txt is a plain Markdown file at your site's root that gives AI language models a curated map of your most useful pages: what your site is, in one quotable summary, and where its highest-signal content lives, with a one-line description per link. It was proposed by Jeremy Howard of Answer.AI as an open standard, and the thing to understand before anything else: it describes, it does not control. Access control for AI crawlers lives in robots.txt; llms.txt is the guidebook you hand the visitors you have already let in.

This guide covers the format, the honest state of whether it works, how to build one by hand, and the platform-by-platform routes, with our own live file as the worked example.

What is llms.txt?

llms.txt is a proposed standard (llmstxt.org) for a Markdown file served at /llms.txt, written for an LLM reader rather than a crawler. Where a sitemap lists every URL for machines that index, llms.txt curates the fifteen to forty pages that matter for a machine that answers, each with a description that adds context a title cannot. The format is deliberately simple: an H1 with the site name, a blockquote summary (the single most valuable line in the file, because it is the sentence a model can lift to answer "what is this company?"), optional notes, then H2 sections of links grouped by intent, with an Optional section for material a model can skip when context is tight.

llms.txt vs robots.txt

The confusion this topic keeps generating, settled in two sentences. robots.txt is access control: it tells crawlers where they may and may not go, and each AI provider's crawlers check it for their own names. llms.txt is curated description: it tells models that already have access which pages matter and why. A disallow rule in llms.txt does nothing; a page description in robots.txt does nothing. They are neighbours at your site root doing unrelated jobs, and a complete setup uses both deliberately.

Does llms.txt actually work?

The honest answer, which most coverage avoids giving: llms.txt is a proposed standard with growing adoption and no confirmed ranking or citation weight from any major engine. Google has not announced support. OpenAI and Anthropic have not published commitments to it. What is documented is adoption on the supply side, thousands of sites now serve one, including major documentation platforms, and the logic on the demand side is straightforward: AI systems are context-limited, curated maps are cheap to read, and the file exists at a predictable location.

Our position, stated plainly: we ship it, and we shipped ours before writing this guide. The cost is minutes, the downside is none, and the option value is real, because if and when engines lean on it, the sites already serving accurate files inherit the advantage, and if they never do, you have lost a coffee break. Treat it as one lever among several in LLM SEO, not a magic file: it curates your content for machines that can already read you, which means access and rendering still come first.

How to create an llms.txt file

The format, step by step, using our own live file, getwellknown.ai/llms.txt, as the worked example.

1. The H1: your site's name. One line, the only strictly required element. Ours: # Wellknown.

2. The blockquote summary. One or two sentences a model could quote verbatim to answer "what is this site?". Write it like the elevator pitch it is. Ours names the product, the mechanism (crawling as each real AI agent) and the audience in a single sentence.

3. Optional notes. A short prose block for anything a reader needs before the links, pricing, contact, a key caveat. No headings allowed here.

4. H2 sections of links, grouped by intent. Not by URL structure: by what someone would ask about. Ours groups Product, Blog and Optional. Each link takes the form - [Page name](url): what this page is useful for, and the description after the colon should add information, not echo the title, because that description is what tells a model whether the page answers the question in hand.

5. The Optional section. Exactly that heading, reserved by the spec for material a model can skip when context is short. Demote secondary pages here rather than deleting them.

Fifteen to forty links is the sensible range. A three-hundred-line llms.txt has missed the point, which is curation; a sitemap already exists for completeness.

llms.txt by platform

Shopify

No native support at the root; files uploaded in Settings will not serve at /llms.txt, so the reliable route is a small app or an edge redirect to a hosted file. Some themes and apps now offer it directly; check before building.

WordPress

Several SEO plugins now generate one, and Yoast has added llms.txt support; alternatively upload a hand-written file to the site root via your host's file manager. Hand-curated beats auto-generated here, because the value is editorial.

Wix

Wix generates an llms.txt automatically for sites on its platform; review what it produced rather than assuming, since auto-generated descriptions tend to echo titles.

Webflow

Serve it as a static asset with a redirect from /llms.txt, or via your hosting layer; Webflow does not currently expose root file uploads directly.

Framer

Same pattern: a custom redirect or your proxy layer serving the static file at the root path.

Cloudflare

If your site sits behind Cloudflare, a static file in your project or a tiny Worker route serves /llms.txt cleanly. One caution that belongs in the same breath: Cloudflare's own defaults can block the very agents that would read the file, so check your AI crawler policy while you are in the dashboard.

How to validate your llms.txt

Until the dedicated tools mature, validation is a five-point manual check: one H1, present and first; a blockquote summary directly after it; no headings other than H1 and H2; every link in the - [name](url): description shape; and the Optional section, if present, spelled and cased exactly ## Optional. Then the check that actually matters: open the file and ask whether a stranger, human or machine, could understand what your site is and where its value lives from this file alone. We are building a free generator and validator to automate this; until it ships, the manual pass takes five minutes. And if you would rather have it checked properly: the Wellknown audit reviews your llms.txt as one of dozens of checks, alongside the layers that decide whether it matters at all, what each AI agent is served, what survives rendering, and whether your best pages are extractable once an agent arrives.

Frequently asked questions

What is an llms.txt file?

A Markdown file at your site's root that gives AI language models a curated, described map of your most valuable pages, plus a one-sentence summary of what your site is. It is descriptive, not a set of access rules.

Does llms.txt actually work?

No engine has confirmed ranking or citation weight for it, and adoption on the reading side is unproven. It costs minutes, carries no downside, and holds real option value, which is why we serve one ourselves while being honest that it is a bet, not a guarantee.

What is the difference between robots.txt and llms.txt?

robots.txt controls access: which crawlers may visit which paths. llms.txt curates description: which pages matter and why, for models that already have access. Rules in one have no effect in the other.

How do I make an llms.txt file?

Write a Markdown file: H1 site name, blockquote summary, optional notes, then H2 sections of links with one-line descriptions and an Optional section for secondary material. Save it as llms.txt at your site root and keep it updated as pages ship.