What Is llms.txt and Does Your Website Need One?
llms.txt is a proposed convention: a Markdown file at your site's root (/llms.txt) that gives AI tools a short summary of the site and a curated list of links to clean, text-friendly pages. It was proposed in 2024 and is popular for software documentation. It is not an official web standard, and Google says you do not need AI text files to appear in its AI features. For most small business sites it is optional: cheap to add, but no substitute for clear, crawlable pages.
- llms.txt is a community proposal published at llmstxt.org, not a W3C or IETF standard.
- Google says no new machine-readable files or AI text files are needed for AI Overviews or AI Mode.
- It does not control crawling; robots.txt does that.
- It is most useful for documentation sites that developers' AI coding tools read.
- If you add one, keep it short, accurate and updated, like any public page.
What llms.txt Is
The llms.txt proposal, first published in 2024 by Jeremy Howard of Answer.AI and now in a second version, suggests that websites place a Markdown file at /llms.txt. The file is meant to help AI agents and language models find the most useful information on a site quickly, without wading through navigation, ads and scripts.
The format is simple. The only required element is an H1 with the site or project name. After that, the proposal describes an optional blockquote summary, optional notes, and sections under H2 headings that list links with short descriptions. A section titled "Optional" marks links an agent can skip when it needs to save space. The proposal also suggests offering clean Markdown versions of important pages at the same URL with .md appended.
A minimal file for a local business might contain the business name, a two-sentence summary of what it does and where, and links to the services, pricing, service-area and contact pages.
What It Is Not
- Not a standard. It is a community proposal. No standards body such as the W3C or IETF has adopted it.
- Not a crawler control. It does not block or allow anything. To control which bots can crawl your site, you still use robots.txt.
- Not a ranking factor anyone has confirmed. No major search engine or AI assistant has publicly said it uses llms.txt to decide what to rank, cite or recommend.
- Not a sitemap replacement. XML sitemaps remain how you tell Google and Bing about your URLs.
What Google and the AI Companies Say
Google is the clearest. Its documentation on AI features and your website states that you do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode. Standard SEO practices apply.
OpenAI's published guidance for site owners focuses on robots.txt and its crawlers: allowing OAI-SearchBot is what it recommends for appearing in ChatGPT search. It does not describe llms.txt as an input to ChatGPT search.
At the same time, several AI companies publish llms.txt files for their own developer documentation. OpenAI's API docs, for example, point readers to an llms.txt index. That reflects the format's real home: documentation that developers' coding assistants read. It is evidence that the format is useful for that purpose, not that it influences how businesses are recommended in chat answers.
There is a practical reason for the caution. A file that a site owner writes about itself is easy to fill with claims the rest of the site does not support, which is why search engines have historically relied on what they can crawl and cross-check rather than on self-declared summaries. The old keywords meta tag followed a similar path: easy to stuff, so search engines stopped trusting it. Whatever role llms.txt eventually plays, AI systems will still verify your claims against your actual pages and what others say about you.
Should Your Business Website Have One?
Good fit
- Software companies, APIs and SaaS products with documentation that developers query through AI tools.
- Sites with large, complex content where a curated map genuinely helps, such as policy libraries or course catalogs.
Low priority
- A typical local service business, practice or shop. The time is better spent on clear service pages, an accurate Google Business Profile, reviews and making sure AI crawlers are not blocked.
Adding one is low risk and takes under an hour, so there is no harm in having a short, accurate file. The risk is treating it as a strategy. If a vendor sells llms.txt as a way to "rank in ChatGPT", ask them which platform has confirmed using it.
If You Do Create One
- Start with an H1 of your business name and a one or two sentence blockquote summary: what you do, for whom, where.
- Add H2 sections such as Services, Company and Contact, each listing links with a short note.
- Link only to pages that exist, are public and are indexable.
- Keep facts consistent with your website and Business Profile; an outdated file is worse than none.
- Update it when services, locations or key pages change, ideally from the same source that builds your sitemap.
Then put your effort where it is known to matter. Our guides on writing content AI search engines cite and getting recommended by ChatGPT cover the work that moves the needle.
Sources and Further Reading
We link to the primary documentation so you can verify the details yourself.
- The /llms.txt file, llmstxt.org
- AI features and your website, Google Search Central
- Overview of OpenAI Crawlers, OpenAI
- Introduction to robots.txt, Google Search Central
Related Questions
Is llms.txt the same as robots.txt?
No. robots.txt is a long-established convention, documented by search engines and standardized by the IETF, that tells crawlers which URLs they may fetch. llms.txt is a proposed guide to your most useful content for AI tools. It grants or blocks nothing, so you still need robots.txt for crawler control.
Will llms.txt help my site appear in Google AI Overviews?
Google says no. Its AI features documentation states that site owners do not need AI text files or new machine-readable files to appear in AI Overviews or AI Mode. Being indexed, eligible for a snippet and genuinely helpful is what qualifies a page.
What is llms-full.txt?
Some sites, mostly documentation platforms, publish a second file containing the full text of their docs in one Markdown document so an AI tool can load everything at once. It is a common companion practice rather than a required part of the proposal, and it is rarely relevant for business websites.
Can llms.txt stop AI companies from using my content?
No. It is an invitation, not a restriction. To limit AI training, use the robots.txt tokens that companies document, such as GPTBot for OpenAI or Google-Extended for Gemini training, and review each company's crawler policy for what it does and does not cover.