Technical Concepts
LLMs.txt

LLMs.txt

llms.txt is a proposed Markdown file at a website’s root. It gives AI agents and language models a short summary of the site and links to its most useful content.
June 27, 2026
Cihan Geyik
Table of Content

A coding agent that needs your API docs, or an assistant answering a question about your pricing, has to find the right page fast. HTML full of menus, scripts and cookie banners slows that down. A file called llms.txt offers a shortcut: one Markdown file that says what the site is and where the useful content lives. It’s cheap to publish, and it’s easy to overrate.

llms.txt is a proposal, not a standard. Many sites publish one, but Google says Google Search doesn’t use it, and no AI platform has committed to ranking content by it.

What is llms.txt?

llms.txt is a file format proposed by Jeremy Howard in September 2024. The llmstxt.org proposal describes its purpose as providing information to help agents use a website.

The file sits at /llms.txt, either at the site root or under a subpath such as /docs/llms.txt. When more than one file applies, the current version of the proposal says agents should use the most specific one.

The proposal also asks sites to offer Markdown versions of useful pages at the same URL with .md added. Agents can then read clean text without parsing HTML.

What does an llms.txt file look like?

The format is plain Markdown with a fixed order. Only the H1 is required.

# Example Co

> Example Co makes invoicing software for freelancers and small agencies.

Plans are billed monthly per user. The API uses token authentication.

## Docs
- [Quick start](https://www.example.com/docs/quick-start.md): Send your first invoice
- [API reference](https://www.example.com/docs/api.md): Endpoints and authentication

## Product
- [Pricing](https://www.example.com/pricing.md): Plans and limits
- [Integrations](https://www.example.com/integrations.md): Accounting and payment tools

## Optional
- [Changelog](https://www.example.com/changelog.md)
  • An H1 with the name of the site or project.
  • A blockquote with a one-line summary.
  • Optional paragraphs or lists with more detail, but no headings.
  • H2 sections that list links in the form [name](url), each with an optional note after a colon.
  • A section named Optional for links an agent can skip when it needs less context.

llms.txt vs robots.txt vs sitemap.xml

FileWho reads itWhat it does
robots.txtCrawlers, before fetchingGrants or denies access to paths
sitemap.xmlSearch engine crawlersLists every URL you want found
llms.txtAI agents and LLM tools, on demandGives a curated summary and links to the best content

The difference matters. llms.txt can’t block a bot or invite one in. A sitemap lists everything, while llms.txt picks the few pages that explain the site best.

Who uses llms.txt?

The proposal’s site says thousands of sites publish the file. They include the developer docs of OpenAI, Anthropic and Google’s Gemini.

Tooling has followed. Documentation platforms such as Mintlify and GitBook generate the file, as do SEO plugins like Yoast SEO and AIOSEO, and Wix. The proposal also notes that Chrome’s Lighthouse checks for one in its agentic browsing audits.

We publish one at ansvisor.com/llms.txt. It has 11 sections, from platform features to the glossary, with short notes on each link.

Does llms.txt improve AI visibility?

There’s no public evidence that it does. Google’s guide to optimizing for generative AI search is direct about it. You don’t need new machine-readable files, AI text files or Markdown to appear in Google Search, because Search doesn’t use them.

That rules out a direct effect on AI Overviews and AI Mode. For other platforms, no vendor has said it ranks or cites pages based on llms.txt.

The realistic benefit is narrower. AI agents and coding tools that fetch your docs can find the right page with fewer steps. For a developer product, that’s worth an hour of work. For a local bakery, it’s optional.

Don’t swap fundamentals for a file. Crawler access, pages that answer real questions and accurate schema markup come first.

How to create an llms.txt file

Pick your most useful pages → Write a one-line summary → Group links under H2 sections → Publish at /llms.txt → Add Markdown versions of top pages
  1. List 10 to 40 pages that answer the questions people ask about you: product, pricing, docs, comparisons and policies.
  2. Write the H1 and a blockquote that says what the company does, in one sentence.
  3. Group the links under H2 sections and add a short note to each.
  4. Upload the file to the site root. Many CMSs and doc platforms can generate it for you.
  5. Review it whenever you launch or retire a major page.

Several open-source projects generate llms.txt along with JSON-LD and robots.txt rules. Our list of open-source AEO and GEO tools for developers covers a few.

Common llms.txt mistakes

  • Dumping every URL from the sitemap. The point is curation.
  • Linking to pages that robots.txt blocks or that require a login.
  • Letting the file go stale after a redesign, so links return 404s.
  • Expecting a ranking lift and skipping the work that earns citations.

How Ansvisor helps

Ansvisor’s Site Audit checks for an llms.txt file alongside robots.txt and AI bot access, as part of 47 weighted signals for any public URL. The file counts as one signal among many, which matches the evidence above.

llms.txt, llms.txt file, LLMs txt, LLM text file, llms.txt proposal

Let AI Search Work for Your Growth

Each prompt you win brings more mentions, more citations and more buyers. Ansvisor keeps finding the next one.

AI Answers Mentions Citations Visibility Score AI Share of Voice AI Traffic Revenue
Win Back Search VisibilityTry Ansvisor 14-day Free →✓ Zero risk. Cancel anytime.

FAQ

Frequently asked questions.

What is llms.txt?

llms.txt is a proposed file, written in Markdown and placed at /llms.txt, that summarizes a website for AI agents. It lists the site’s most useful pages with short notes, so an agent can find answers without reading every page.

Is llms.txt an official standard?

No. Jeremy Howard proposed it in September 2024, and llmstxt.org still calls it a proposal open to community input. Thousands of sites publish one, but no standards body has adopted it.

Does llms.txt help SEO or AI visibility?

Google says Google Search doesn’t use llms.txt and you don’t need AI text files to appear in its results. There’s no public evidence that it raises citations elsewhere. Its value is helping agents and developer tools read your content.

What should an llms.txt file include?

Start with an H1 naming the site, then a one-line blockquote summary. Add H2 sections with Markdown links to your most useful pages, each with a short note. Put links an agent can skip under a section called Optional.

What is the difference between llms.txt and robots.txt?

Robots.txt tells crawlers which URLs they may fetch. llms.txt doesn’t grant or block anything. It’s a curated guide that agents read when they need information about your site.

Help us grow the AI Visibility Glossary

New terms are added regularly.

Help us improve the page or suggest a new term →
About the Author
Cihan Geyik

Cihan Geyik

Co-founder at Ansvisor

Cihan Geyik is the co-founder of Ansvisor, an open-source AI Visibility platform for AI Search. With more than 15 years of experience in digital marketing and growth, he writes about AI visibility, AI search, AEO, GEO, citations, and answer engines. He focuses on helping brands understand and improve their presence across ChatGPT, Gemini, Google AI Overviews, Bing AI Mode and other AI answer engines.

Google Preferred Sources
Get more Ansvisor insights in Google

Add Ansvisor as a preferred source and Google will show you more of our AI search, AEO and GEO guides.