llms.txt: what it is, what the evidence shows, and how to write one

What is llms.txt designed to do?

llms.txt is a proposed convention for a Markdown file at a website's root that highlights selected pages for language-model tools. Its purpose is to make a concise guide to useful material available, rather than to replace the pages themselves or control how a model responds. The proposal and format describe the idea; treat it as a convention, not a universal web standard.

A site owner can use the file to point readers or compatible tools toward clear, substantive resources: product documentation, policies, technical references, or well-maintained explainers. A curated list can be easier to scan than a large site navigation, especially where important information is spread across many pages. That is a usability rationale, not proof that a particular AI system will read or prioritize the file.

The useful question is not whether every site needs llms.txt. Ask whether you can name a small set of pages that explain the organization or product accurately, and whether a simple text index would help someone find them. If the list would be redundant, stale, or hard to maintain, publishing the file may add little value.

What evidence shows that llms.txt affects AI answers?

The proposal explains a possible way to make selected content easier to discover; it does not establish that AI systems use the file, cite its pages, or change their answers because it exists. Do not present llms.txt as a confirmed ranking signal or as a route to inclusion in ChatGPT, Perplexity, or Google results.

Before making a claim about impact, distinguish implementation from outcome. You can verify that a file is published at the intended URL, that it returns successfully, and that its links lead to the intended pages. You can also observe whether specific AI answers mention your site over time. Those observations do not by themselves show that llms.txt caused a change: answers can vary, sources can change, and a file may be ignored.

A practical evidence standard is to document what was changed and what was measured. Keep the file version, publication date, selected URLs, and a sample of relevant prompts or queries. Compare the same questions and engines before and after, and record citations as observations rather than proof of causality. For a broader technical review, see technical AEO: schema, llms.txt, and crawlers and AI visibility monitoring.

Is llms.txt needed for your website?

Most sites can decide by weighing the likely maintenance cost against a clear information-organization benefit. The file is more reasonable to test when a site has important material that is difficult to locate, and someone can keep its links accurate. It is not a substitute for publishing useful pages or making them available to visitors and search crawlers.

Use this quick decision check:

• Create a draft if your documentation, product details, or policies are valuable but scattered across the site.
• Keep the scope small if a few canonical pages can represent the material accurately.
• Wait if your site is changing rapidly, key pages are not ready, or no owner will review the list.
• Skip it for now if the only goal is to improve rankings or force AI citations; the file cannot promise either outcome.

For Web3 projects, candidate pages might include product documentation, chain or token information, security disclosures, governance material, and support guidance. Include only pages that are public, current, and consistent with the facts shown elsewhere. A file cannot repair contradictory claims or unclear project identity. If your priority is improving the underlying AI-search foundations, compare this task with the wider work described in AI search visibility and AI SEO.

LLMs.txt vs schema.org, robots.txt, and sitemaps

llms.txt, schema.org markup, robots.txt, and XML sitemaps serve different purposes; using one does not make the others unnecessary. The distinction matters because a text index is easy to mistake for a technical control that it does not provide.

Schema markup expresses meaning in a structured vocabulary, while llms.txt offers a human-readable selection of links. Use schema.org when you have accurate, relevant structured data to add, and consult Google's search documentation for current guidance on search features and crawling. Do not add schema just to repeat a file's list of URLs.

For a more detailed comparison of structured data and AI search, read schema markup for AI search. A sound implementation can use these tools together, but each needs a clear job: accurate content, sensible access, useful discovery paths, and structured descriptions where appropriate.

How to implement llms.txt without creating a maintenance problem

A useful implementation starts with page selection and ownership, not with generating a file. First decide what a reader needs to understand about your site, then choose the smallest set of authoritative pages that answers that need. Publish the file at the expected root location and check that its contents and links are readable.

Use this implementation sequence:

1. Inventory candidate pages. Gather canonical product, documentation, policy, and support URLs. Exclude drafts and duplicate explanations.
2. Check access and accuracy. Open each URL without special permissions, confirm it loads, and compare its claims with the current product and public disclosures.
3. Write a short introduction. Explain what the site or project is in plain language. Avoid unsupported claims and promotional filler.
4. Group and label links. Use headings that describe the resource type. Give each page a brief, accurate description so its purpose is clear.
5. Publish and validate. Put the file at the domain root, check the response and Markdown formatting, and test every link.
6. Assign a review owner. Revisit the file when a listed page moves, changes materially, or becomes obsolete.

Coordinate with whoever manages your website or deployment so the file does not get overwritten by a later release. Keep a copy of the previous version and note the reason for substantial edits. For related discovery and content work, the technical AEO guide can help you assess how llms.txt fits alongside crawler access and structured data.

How to write an llms.txt file that is actually useful

Write the file for a person trying to understand the site quickly. A plain-language introduction followed by a few well-labeled groups is easier to assess than an exhaustive inventory of every URL. The file should direct attention to pages with distinct, lasting value, not duplicate entire sections of the site.

For each candidate page, ask:

• Is this the clearest source for the topic, rather than an outdated or secondary copy?
• Does the page state its scope, audience, and key terms without relying on missing context?
• Are important facts, including product status and risk disclosures, explicit and consistent?
• Can a visitor access the page, and does its URL remain stable?
• Would the description still be accurate if read outside the surrounding site navigation?

Use literal headings such as “Documentation,” “Security,” or “Policies.” Keep descriptions factual: identify the subject and what a reader will find, rather than claiming the page is definitive or the project is the best. Avoid listing pages simply because they contain keywords. For a crypto project, prioritize materials that help verify what the project does and how it works; do not use the file to make claims that are not supported on the linked pages.

There is no benefit in making the file long for its own sake. A smaller list that someone can maintain is usually a better editorial choice than an ambitious directory that becomes stale. Treat each link as a commitment to keep the destination useful.

How to evaluate an llms.txt test over time

Evaluate llms.txt as a documented technical and editorial test, not as a campaign with a promised citation outcome. Start by saving the current version of the file and recording the pages it names. Check that the file is available at the root, its links work, and the linked material says what the file claims it says.

If you want to observe AI answers, choose a small set of real questions that reflect how a prospective user might describe the product or topic. Save the exact wording, engine, date, answer, and cited sources. Repeat the same prompts later and note differences without treating a single answer as a reliable trend. Separately check ordinary site analytics or server logs only where you have access and can interpret them responsibly; a visit does not necessarily reveal why a system fetched a page.

When results are unclear, review the foundations before adding more links. Are key pages accessible and clear? Does the site describe the same entity consistently? Are important facts supported by the pages you selected? A technical review can be paired with AI visibility monitoring or work on getting cited in ChatGPT, but those activities measure different things. Record the limits of your test so stakeholders do not mistake correlation for proof.

What llms.txt cannot control

Publishing llms.txt gives you control over the file you maintain, not over how an outside system processes it. A crawler or AI product may not fetch it, may interpret it differently, or may rely on other sources when generating an answer. Search systems also set their own crawling and presentation rules; a site owner cannot use this proposed convention to require indexing, ranking, or citation.

That distinction is especially important when the site contains fast-changing information. A list can point to a page, but it cannot make the page accurate, resolve conflicting token or project details, or replace a clear update process. The team remains responsible for the linked content, permissions, disclosures, and any applicable platform or search policies. Check official guidance for the specific systems you care about rather than assuming that a new file changes their rules.

Our work can cover an agreed review, file draft, link validation, and implementation coordination. What we cannot promise is that a named AI service will read the file, cite a page, change a description, or show a particular answer. Keep that boundary explicit in your project brief. If a proposed benefit depends on a platform action, describe it as something to observe, not a deliverable.

Where llms.txt fits in a broader AI-search plan

llms.txt is a small optional component of AI-search readiness, not a standalone strategy. Start with the sources that an answer engine or a human visitor could actually assess: clear pages, stable facts, accessible documentation, and a recognizable organization or product. Add a curated file only when it makes those sources easier to navigate.

A sensible order is to clarify the pages that explain the project, review factual consistency, check crawl access, and use structured data where it accurately describes the page. Then decide whether an llms.txt file adds a useful index. This order prevents teams from polishing a directory before the destinations are ready. For a wider technical overview, see technical AEO; for the broader discipline, explore AI search visibility.

Keep the first version simple enough to review during normal site maintenance. Name an owner, document why each page is included, and remove links when their destinations stop serving the stated purpose. That gives you an auditable file and a clear basis for deciding whether to keep it. The measure of success is a sound implementation and a maintainable set of references, not an assumed effect on a particular model.

Prices

Starting prices in USD. Custom bundles and volume discounts on request. Payment in USDT, USDC, BTC, ETH, SOL, TON or your project token.

How it works

1. Choose the goalDecide whether the file solves a real navigation problem. Do not define success as a guaranteed ranking or citation.
2. Select authoritative pagesChoose a short list of public, current pages that explain the site, product, policies, or documentation.
3. Write clear descriptionsGroup links under literal headings and describe each destination accurately, without promotional claims.
4. Publish and validatePlace the file at the root location, check formatting and access, and test each listed URL.
5. Review and recordAssign an owner, note material changes, and measure any AI-answer observations separately from implementation checks.