llms.txt
llms.txt is a proposed standard for a plain text file, written in Markdown, placed at the root of a website (for example, yourdomain.co.uk/llms.txt). It gives large language models and AI tools a concise overview of the site and links to its most important content in a format that is easy for them to read.
Where llms.txt came from
The idea was proposed in 2024 by Jeremy Howard of Answer.AI. Web pages are full of navigation, scripts, adverts and layout that make them harder for AI tools to process within limited context windows. A short, curated Markdown file can point AI tools straight to the information that matters.
What an llms.txt file contains
The proposal suggests a simple structure:
- An H1 heading with the name of the site or project.
- A short summary in a blockquote describing what the site is about.
- Optional details in plain paragraphs or lists.
- Sections with H2 headings containing lists of links to key pages, each with a brief description.
- An optional section for less important resources that can be skipped if space is limited.
Some sites also publish llms-full.txt, which includes the full text of key documentation in one file.
How llms.txt differs from robots.txt and sitemaps
- Robots.txt tells crawlers what they may or may not access.
- XML sitemaps list URLs for search engines to crawl.
- llms.txt curates and explains the most useful content for AI tools to read and understand.
They serve different purposes and can be used together.
Does llms.txt help SEO or AI visibility?
Currently, the benefit is uncertain. Major AI providers have not confirmed that they use llms.txt when crawling or answering questions, and Google has said it does not use it for search. Some developer tools and AI coding assistants do read llms.txt files, especially for technical documentation.
In short, llms.txt is low effort and low risk, but it is not a substitute for the fundamentals of AI SEO: crawlable pages, clear content, strong authority and accurate information across the web.
Who might benefit most
- Software companies and API providers with technical documentation.
- Sites with large knowledge bases or reference content.
- Businesses that want to experiment with emerging AI standards.
For a small business website, adding a simple llms.txt file listing your main services, about page, contact details and key guides takes minutes and does no harm.
How to create an llms.txt file
- Write a short Markdown file with your site name, a one-paragraph summary and lists of key pages with descriptions.
- Upload it to the root of your domain.
- Keep it updated when important pages change.
- Make sure the linked pages are accessible to AI crawlers in your robots.txt settings.
Treat llms.txt as an experiment, and judge its value by watching AI referral traffic and mentions over time. See also ChatGPT Search and Generative Engine Optimisation (GEO).
Further reading: the llms.txt proposal.
Key Takeaways
- llms.txt is a proposed Markdown file summarising your site for AI tools.
- It complements, but does not replace, robots.txt and sitemaps.
- Major AI providers have not confirmed using it for search.
- It is quick and low-risk to add.
- The fundamentals of AI SEO matter far more.
Related Terms
Want help putting this into practice? Get a free SEO check, or keep learning in the SEO Encyclopedia.