# What is llms.txt and should your site have one

What llms.txt is, what goes in it, a real example, which AI tools say they read it, and how to add one to your site in about ten minutes.

Canonical: https://ignura.com/blog/what-is-llms-txt
Last updated: 2026-10-29
Author: Jawad Jalal

llms.txt is a Markdown file at yourdomain.com/llms.txt that tells AI agents what your site is and links them to the pages worth reading. It is a proposal, not a standard: Google says it isn't needed for Google Search, and no major AI company has said its crawlers or assistants read the one on your site.

So should you have one? If it takes an afternoon or less, yes. It is cheap, it can't hurt your Google rankings, and agents that do look for it get a clean map of your site. Just don't expect it to get you cited on its own.

## The short version

- llms.txt was proposed by Jeremy Howard on 3 September 2024. Version 2 of the proposal was published on 10 August 2026 ([llmstxt.org](https://llmstxt.org/)).
- The format is plain Markdown: a `#` title, a short summary in a blockquote, then `##` sections that list links with a line of notes each.
- It isn't needed for Google Search. Google says the file "will neither harm nor help your site's visibility or rankings in Google Search", and that you don't need AI text files to appear in Google Search, its generative AI features included ([Google](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide)).
- OpenAI, Anthropic, Perplexity and Google publish llms.txt files for their own developer docs. Their crawler docs talk about robots.txt, not llms.txt.
- Chrome's Lighthouse checks for the file in its experimental Agentic Browsing category ([Chrome for Developers](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt)).
- To make one, run your sitemap through the [llms.txt generator](https://ignura.com/tools/llms-txt-generator), cut it down to the pages that matter, and write a real line for each.

## What goes in an llms.txt file

The [llmstxt.org proposal](https://llmstxt.org/) sets out the sections in a fixed order:

1. An H1 with the name of the site or project. This is the only required part.
2. A blockquote with a short summary, holding what someone needs to make sense of the rest.
3. Optional paragraphs or lists with more detail. No headings here.
4. Optional H2 sections, each a "file list" of links written as `[name](url)`, with a colon and a note after the link if you want one.

A section called `## Optional` has a special meaning. It holds secondary links an agent can skip when it is short on room.

The bare skeleton looks like this:

```markdown
# Your product

> One or two sentences: what it is, who it is for, what it does.

## Docs

- [Quick start](https://example.com/docs/quickstart.md): Set up in five steps
- [API reference](https://example.com/docs/api.md): Every endpoint and its parameters

## Optional

- [Changelog](https://example.com/changelog.md)
```

Version 2 adds a few things. A file can sit at a subpath like `/docs/llms.txt` and cover only the pages under it. Pages can offer a Markdown copy at the same URL with `.md` on the end. And the proposal suggests pointing to both with standard link tags: `rel="alternate" type="text/markdown"` for the Markdown copy and `rel="describedby"` for the llms.txt that covers the page.

## A real example: ignura.com/llms.txt

Our own [llms.txt](https://ignura.com/llms.txt) follows the format. Trimmed, with the summary shortened, the shape is:

```markdown
# Ignura

> A GEO and SEO agency for B2B SaaS and devtools.

## Who it's for
## The search problems we fix
## Pages
## Kindling (blog posts)

- [How to track ChatGPT traffic in GA4](https://ignura.com/blog/track-chatgpt-traffic-in-ga4): Find ChatGPT, Perplexity and Gemini visits in GA4, build an AI assistants channel group, tie them to sign-ups, and see what the numbers miss.

## Free tools in Markdown

- [AI crawler checker](https://ignura.com/tools/ai-crawler-checker.md)

## Optional

- [Full facts and the full text of every post](https://ignura.com/llms-full.txt)
- [Kindling RSS feed](https://ignura.com/blog/feed.xml)
```

Three things in it are worth copying:

- Every post and page link has a one-line description, so an agent can pick the right page without opening all of them.
- Each post also has a Markdown copy (for example [this one](https://ignura.com/blog/track-chatgpt-traffic-in-ga4.md)), and each HTML page points to its copy with `rel="alternate" type="text/markdown"`.
- The long version lives in a separate file, linked under Optional, so the main file stays short.

Our build regenerates the file every time we publish, so it stays in step with the site. A hand-written llms.txt goes stale fast.

## llms.txt vs llms-full.txt vs robots.txt vs sitemap.xml

These four files get mixed up a lot. They do different jobs.

| File | What it does | Who reads it |
| --- | --- | --- |
| robots.txt | Says which URLs crawlers may fetch ([Google](https://developers.google.com/search/docs/crawling-indexing/robots/intro)) | Search engines and AI crawlers that follow it |
| sitemap.xml | Lists the URLs you want search engines to crawl ([Google](https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview)) | Search engines |
| llms.txt | A short, hand-picked map of your site with notes, for agents | Agents and tools that look for it. Not needed for Google Search |
| llms-full.txt | The full text of your key pages in one file | Agents and tools that look for it |

The key point: llms.txt doesn't allow or block anything. If GPTBot or PerplexityBot is blocked in robots.txt, an llms.txt won't let it in. Check that first with the [AI crawler checker](https://ignura.com/tools/ai-crawler-checker).

llms-full.txt is not part of the llmstxt.org proposal. It is a convention that docs platforms such as [Mintlify](https://www.mintlify.com/docs/ai/llmstxt) generate alongside llms.txt, and Perplexity publishes one for [its own docs](https://docs.perplexity.ai/llms-full.txt). Make one if your key pages fit in a single file of reasonable size. Skip it if they don't.

The proposal also explains why a sitemap is not a substitute. A sitemap often won't list Markdown copies, can't point to useful pages on other sites, and usually covers far more than fits in a model's context.

## Who reads llms.txt

This is where most articles overclaim. Here is what the companies themselves say, checked on 28 October 2026.

**Google Search: not needed.** Google's guide to its generative AI features lists llms.txt under things you can ignore for Google Search, and says creating one "will neither harm nor help your site's visibility or rankings in Google Search" ([Google](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide)). Google has not committed to using it.

**Chrome's Lighthouse: it checks for one.** Lighthouse has an experimental Agentic Browsing category with an llms.txt audit. A missing file is marked not applicable, since the file is "optional at the moment" ([Chrome for Developers](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt)). If you have one, the [audit's code](https://github.com/GoogleChrome/lighthouse/blob/main/core/audits/agentic/llms-txt.js) checks for an H1, at least one Markdown link and at least 50 characters. That is a readiness check, not a ranking signal.

**OpenAI, Anthropic and Perplexity: no public commitment.** Their crawler pages ([OpenAI](https://developers.openai.com/api/docs/bots), [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), [Perplexity](https://docs.perplexity.ai/guides/bots)) explain how to control their bots with robots.txt. None of them says their crawlers or assistants read llms.txt on other people's sites.

**They do publish one.** OpenAI ([developers.openai.com/llms.txt](https://developers.openai.com/llms.txt)), Anthropic ([platform.claude.com/docs/llms.txt](https://platform.claude.com/docs/llms.txt)), Perplexity ([docs.perplexity.ai/llms.txt](https://docs.perplexity.ai/llms.txt)) and Google's Gemini API ([ai.google.dev/gemini-api/docs/llms.txt](https://ai.google.dev/gemini-api/docs/llms.txt)) all have one for their developer docs. Publishing a file is not the same as reading yours. It does show where the format is used most: docs that coding agents fetch while someone writes code. The proposal's author says the same, that llms.txt is used most heavily for software documentation. If you sell to developers, [SEO and GEO for developer tools](https://ignura.com/devtools) covers the docs side.

**Does it get you cited?** There's no evidence it does. SE Ranking looked at nearly 300,000 domains, found an llms.txt on 10.13% of them, and found no link between having the file and how often AI models cited the domain ([SE Ranking](https://seranking.com/blog/llms-txt/)). That is one correlation study from a tool vendor, so it doesn't prove the file can never help. It does mean you shouldn't skip the work that is known to matter.

## How to add llms.txt to your site

1. **Check your crawler access first.** Run your domain through the [AI crawler checker](https://ignura.com/tools/ai-crawler-checker). An llms.txt is no use to a bot that robots.txt keeps out.
2. **Generate a first draft.** Paste your sitemap.xml into the [llms.txt generator](https://ignura.com/tools/llms-txt-generator). You get the H1, blockquote and H2 sections in the right format.
3. **Cut it down.** Keep the pages a buyer or an agent needs: what you do, who it's for, pricing, docs, key guides. Move the rest under `## Optional` or drop it.
4. **Write a real line for each link.** "Pricing" tells an agent nothing. "Plans, what each includes, and how billing works" tells it when to open the page.
5. **Put it at the root.** Serve it at `/llms.txt` as plain text and make sure it returns a 200. If your docs live under `/docs/`, a second file at `/docs/llms.txt` can cover just those.
6. **Add Markdown copies if you can.** Clean `.md` versions of key pages, linked from llms.txt and from each page's `<head>`, give agents text without the navigation and scripts.
7. **Check it.** Paste the finished file back into the [checker](https://ignura.com/tools/llms-txt-generator) or run Lighthouse's Agentic Browsing audit. Then do the test the proposal suggests: give an AI assistant only your llms.txt and ask it questions about your product.
8. **Keep it current.** Generate it from your build or CMS if you can. Yoast SEO, AIOSEO, Wix, GitBook and Mintlify can all create one, per the [proposal's list](https://llmstxt.org/).

The first draft takes about ten minutes. The descriptions are where the time goes, and where the value is.

## What matters more than llms.txt

If you have one afternoon for AI visibility, spend most of it elsewhere:

- **Let the crawlers in.** Allow OAI-SearchBot, PerplexityBot and Claude's bots in robots.txt, and check your CDN or firewall isn't blocking them.
- **Answer in the first lines.** Pages that answer the question right away are easier to quote. The [guide to getting cited in AI answers](https://ignura.com/blog/how-to-get-cited-in-ai-answers) covers this step by step.
- **Be on the pages AI already quotes.** For many buyer questions, the answer cites review sites, lists and threads, not your homepage. That is the [citations layer](https://ignura.com/citations).
- **Fix the technical basics.** Pages that aren't indexed or that need JavaScript to show their text are harder for every crawler to read. That is [technical SEO](https://ignura.com/technical-seo).
- **Measure it.** Track AI referrals and sign-ups, as in [how to track ChatGPT traffic in GA4](https://ignura.com/blog/track-chatgpt-traffic-in-ga4).

llms.txt sits at the end of that list. It's a small, cheap step that helps agents read your site. It is not a shortcut into the answers.

## FAQ

### Is llms.txt an official standard?

No. It is an open proposal from Jeremy Howard, first published on 3 September 2024 and updated to version 2 on 10 August 2026 ([llmstxt.org](https://llmstxt.org/)). There is no standards body behind it, and anyone can comment on it through its public GitHub repository.

### Does Google use llms.txt?

Not for Search. Google says having one "will neither harm nor help your site's visibility or rankings in Google Search", and that you don't need AI text files to appear in its generative AI features. Chrome's Lighthouse does check for the file in an experimental audit for agent readiness, which is a separate thing.

### Do ChatGPT, Claude and Perplexity read llms.txt?

None of them has said so publicly. Their crawler docs cover robots.txt. All three companies publish an llms.txt for their own developer docs, which is not the same as reading yours. SE Ranking notes that some SEO logs show GPTBot fetching llms.txt files now and then ([SE Ranking](https://seranking.com/blog/llms-txt/)). Your own server logs will show whether any bot requests yours.

### Where does llms.txt go?

At the root of your site, at `/llms.txt`, served as plain text. Version 2 of the proposal also allows a file at a subpath, such as `/docs/llms.txt`, covering only the pages under it.

### Can llms.txt block AI crawlers?

No. llms.txt points agents at content. Blocking and allowing crawlers is done in robots.txt, and at your CDN or firewall.

### Does llms.txt help SEO?

Not in Google Search: Google says it "will neither harm nor help" your visibility there. Its use is helping agents and coding tools find the right pages, mostly on docs sites.

## Want to know where you stand in AI answers?

llms.txt is one small part of the [technical layer](https://ignura.com/technical-seo). The bigger question is whether ChatGPT, Perplexity and Google's AI name you when buyers ask about your category. How we price that work is on the [pricing page](https://ignura.com/pricing).

If you want a second opinion on how AI tools see your site, we'll go through it with you on a free 15-minute call.

[Book your free Growth Check](https://ignura.com/book?utm_source=site&utm_medium=blog&utm_content=what-is-llms-txt)

Book a 15-minute call about your website: https://ignura.com/book