What is llms.txt?
llms.txt is a small Markdown file at the root of your website that tells AI assistants what your site is about and where your most useful pages are. Think of it as a short, friendly guide for large language models - robots.txt controls access, llms.txt explains content.
The format was proposed at llmstxt.org: a title, a one-line summary, then sections of links with short descriptions. Here is what one looks like:
# Bright Smile Dental > Family dental clinic in Austin, Texas offering checkups, cleanings, whitening and emergency care. ## Main pages - [Services](https://brightsmile.example/services): General, cosmetic and emergency dental services - [Book an appointment](https://brightsmile.example/book): Online booking for new and existing patients ## Company - [About us](https://brightsmile.example/about): Our dentists, clinic hours and location
How to add llms.txt to your website
- Generate your file above and edit anything that is out of date.
- Download it - the file must be named exactly
llms.txt. - Upload it to your site's root folder, next to robots.txt, using your host's file manager or FTP.
- Open
yoursite.com/llms.txtin a browser to confirm it loads as plain text. - Update it when you add important pages, products, or services.
On WordPress, you can also use an SEO plugin that supports llms.txt. If your website builder does not allow files at the root, upload the file elsewhere and add a redirect from /llms.txt.
Should you let AI crawlers read your site?
- Search and browsing crawlers (OAI-SearchBot, ChatGPT-User, Claude-User, PerplexityBot) fetch pages when someone asks an AI assistant a question. Blocking them can stop AI tools from citing or recommending your business.
- Training crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot) collect content for future AI models. Blocking them is a content-licensing choice and does not affect regular Google search.
- Many small businesses allow both to stay visible as more customers search with AI.
Want visitors on your own website to get the same instant answers? Try a FiveMinChat demo built from your pages, or see our other free tools.
Frequently asked questions
What is llms.txt?
llms.txt is a plain-text Markdown file at the root of a website (yoursite.com/llms.txt) that gives AI assistants a short summary of the site and links to its most important pages. It was proposed in 2024 at llmstxt.org as a simple way to help large language models understand a website.
Does llms.txt help SEO or AI search rankings?
llms.txt is a proposed standard, not an official ranking factor. Google has not said it uses llms.txt, and no AI company guarantees it will read the file. It is quick to add and can help AI tools that do look for it, but it does not replace good page content, a sitemap, or robots.txt.
Where do I upload llms.txt?
Upload it to the root folder of your website so it opens at yoursite.com/llms.txt, the same place as robots.txt. On WordPress you can use your host's file manager or an SEO plugin that supports llms.txt. If your platform does not allow files at the root, host the file elsewhere and add a redirect from /llms.txt.
What is the difference between llms.txt and robots.txt?
robots.txt tells crawlers which pages they may or may not fetch. llms.txt does not block anything - it is a guide that describes your site and points AI tools to your best pages. Use robots.txt to control access and llms.txt to explain your content.
Should I block AI crawlers like GPTBot and ClaudeBot?
It depends on your goals. Blocking training crawlers keeps your content out of future AI training data, but blocking search and browsing crawlers (like OAI-SearchBot, ChatGPT-User, or PerplexityBot) can stop AI assistants from citing or recommending your business. Many small businesses allow them to stay visible.
Is this llms.txt generator free?
Yes. It is free with no signup. We read up to 40 public pages from your sitemap or homepage, and you can edit the result before you upload it.