Do I Need an llms.txt File? What the 2026 Data Says

No, most websites do not need an llms.txt file. Google says Search does not use these files, no major AI provider lists llms.txt as a requirement for being crawled or cited, and the largest studies so far find no link between having the file and getting cited more by AI answers. It is worth publishing, or customizing, only when an AI agent you care about will actually read it and someone will keep it accurate.

This guide is for e-commerce and Shopify teams deciding whether llms.txt deserves a slot on the roadmap. You'll get what the file is, what Google, OpenAI, Anthropic and Perplexity say about it, what the 2026 data shows about who reads it, the twist for Shopify stores (you probably already have one), a decision table and the fixes that matter more for AI visibility.

What is an llms.txt file?

llms.txt is a proposed standard written by Jeremy Howard and first published on September 3, 2024. The proposal was updated to version 2 in 2026 (llmstxt.org shows a last modified date of August 10, 2026). It describes a Markdown file that gives AI agents a short introduction to a site and a curated list of links to the most useful resources. Think of it as a reading guide, not a copy of your site and not a set of rules.

The format

• An H1 with the site or project name. This is the only required part.
• An optional blockquote summary that explains what the site is.
• Optional paragraphs or lists with extra detail, without headings.
• Optional H2 sections with link lists, each link written as [name](url) with an optional note.
• An optional section named "Optional" for secondary links an agent can skip when it needs less context.

The file usually lives at /llms.txt, but v2 also allows files under a subpath such as /docs/llms.txt, where the most specific file covers the URLs under it. The proposal also suggests offering clean Markdown copies of key pages (for example page.md) so agents can read them without HTML clutter.

What it was designed for

The author expected llms.txt to be used mainly at inference time, when an agent fetches content on demand to help a user, rather than for model training. Adoption has been strongest on software documentation sites, where coding agents use the file to find API references. That origin matters: the use case was "help a developer's AI assistant find the right docs page," not "rank a product in ChatGPT."

Do Google and ChatGPT use llms.txt?

For Google, the answer is a clear no. Google's guide to optimizing for generative AI features in Search (updated July 10, 2026) lists llms.txt under things you can ignore. It says you don't need to create AI text files, special markup or Markdown to appear in Google Search, that Search itself doesn't use them, and that maintaining one will neither harm nor help your visibility or rankings. Google's page on AI features and your website says a page only needs to be indexed and eligible to show with a snippet to appear in AI Overviews or AI Mode.

For ChatGPT, OpenAI's crawler documentation explains robots.txt controls for GPTBot, OAI-SearchBot and ChatGPT-User. It does not mention llms.txt. Anthropic's and Perplexity's crawler documentation also explain robots.txt controls, not llms.txt. That does not prove those companies never read the file, but nobody can honestly promise that publishing one improves your visibility in their products.

What the 2026 data says about llms.txt

Two large studies now answer the questions every store owner asks: how many sites have one, does anyone read it, and does it lead to citations?

Adoption is real, but readers are rare

Ahrefs looked at all 137,210 domains in Ahrefs Web Analytics that received traffic in May 2026. Its llms.txt study (June 15, 2026) found that 28% of those domains publish an llms.txt file, and 97% of those files received zero requests in May. The remaining 3%, about 1,100 domains, received all of the measured llms.txt traffic, roughly 22,000 requests, and 96% of those requests came from bots.

Keep the sample in mind: these are sites that installed an SEO vendor's analytics script, so they skew toward marketers who follow SEO trends. That likely explains why adoption is higher than in other samples.

Most bots fetching llms.txt are not AI search bots

The more telling finding is who does the fetching. In Ahrefs' data, SEO audit tools were the largest single category of requests, and Ahrefs reports that 77% of the bots fetching llms.txt are not AI tools at all. AI retrieval bots, the ones that fetch pages to answer a user's question in tools like ChatGPT and Perplexity, made up just 1.1% of requests.

Within the AI categories, GPTBot (OpenAI's training crawler) was the biggest single fetcher, and Claude-Code (Anthropic's coding agent) ranked second among named AI tools. That fits the original design: coding agents reading documentation. It does not suggest shopping assistants are reading store llms.txt files to decide what to recommend.

No measurable link to AI citations

SE Ranking analyzed about 300,000 domains for its llms.txt research and found the file on 10.13% of them. Using Spearman correlation, an XGBoost model and SHAP analysis, it found no correlation between having llms.txt and how often a domain is cited by AI. Removing the llms.txt variable from the model actually improved its accuracy. Adoption was also flat across traffic levels, so big brands are not quietly winning with it.

Both studies are correlational, so they cannot prove the file never helps anyone. But combined with silence from every major AI provider, the burden of proof sits with anyone selling llms.txt as a visibility tactic.

Shopify stores already have an llms.txt

Here is the part most llms.txt articles miss. If you run a Shopify store, the question is not "should I add one?" because Shopify already serves one for you.

According to Shopify's developer changelog (May 28, 2026), every store includes a default agents.md file at /agents.md, and the paths /llms.txt and /llms-full.txt serve the same content by default. When we checked two large Shopify stores, Allbirds and Chemical Guys, on October 27, 2026, both returned a Shopify-generated Markdown file at /llms.txt titled "Agent Instructions." It points shopping agents to the Shop skill and the store's Universal Commerce Protocol (UCP) endpoints for product discovery, carts and checkout.

In other words, Shopify's default file is not a curated reading list of your best content. It is a machine-facing instruction sheet for AI shopping agents, maintained by Shopify.

Should you customize the Shopify default?

Shopify's own llms.txt.liquid documentation says that in most cases you shouldn't add a custom llms.txt template. It recommends keeping the Shopify-managed default unless you have an advanced reason for /llms.txt to differ from /agents.md, and notes that if you add one you must maintain it yourself. A few constraints worth knowing:

• The lookup order is llms.txt.liquid, then agents.md.liquid, then the Shopify default. To change all three agent files at once, edit agents.md.liquid.
• The template runs in a restricted context. Standard objects such as shop, collections and articles are not available, so you can't loop through your catalog the way you would in a normal theme template.
• The file is served on your primary domain only and has no localized versions, so Shopify Markets prefixes don't apply.

What llms.txt cannot do

Most of the confusion around llms.txt comes from expecting it to work like robots.txt or a sitemap. It doesn't.

• It does not block crawling or training. Access rules live in robots.txt. To opt out of training, use each provider's documented control, such as a GPTBot or ClaudeBot rule, or the Google-Extended token. Our guide to llms.txt vs robots.txt covers the difference, and should I block GPTBot? covers the trade-offs.
• It does not fix blocked or broken pages. If product pages return errors, sit behind a bot challenge or load key details only through JavaScript, a guide file cannot get them read.
• It does not protect private content. Everything in it is public. Never list URLs you want kept private.
• It does not override what your pages say. If your return policy says 30 days on one page and 14 on another, a summary file will not resolve that for an AI model.

When adding llms.txt makes sense

The file earns its keep when you have a specific reader and an owner. Use this table to decide.

For a non-Shopify site, our llms.txt generator can draft the file, and our step-by-step guide on how to create an llms.txt file walks through the structure. Review every description and link by hand. Generated text is a starting point, not proof that your business details are correct. Adding the file because a competitor did is not a use case.

What to fix before you touch llms.txt

If your real goal is to show up in AI answers, these steps matter far more than a guide file. Work through them in order.

1. Check your robots.txt. Make sure you are not blocking search crawlers you want, such as Googlebot, OAI-SearchBot, Claude-SearchBot or PerplexityBot. Paste your file into our robots.txt and AI crawler tester to see how each bot is treated, or follow our guide to check if ChatGPT can crawl your website.
2. Check your CDN and firewall. Bot-management settings can challenge or block crawlers even when robots.txt allows them. Look at server or CDN logs for real requests from AI bots and their status codes.
3. Put key content in the HTML. Product names, prices, availability, sizing and policies should be in the page source, not only loaded by JavaScript after render.
4. Remove contradictions. Make shipping times, return windows and product specs match everywhere they appear, including product feeds and marketplace listings.
5. Keep structured data accurate. Product, Organization and breadcrumb markup won't get you into AI answers by itself, but Google still recommends it for rich results. Check it with our schema validator.
6. Answer buying questions on the page. Comparison tables, FAQs and specific "who this is for" copy give AI systems something concrete to quote. Our guide to AI search visibility covers this in depth.
7. Only then, decide on llms.txt. Use the decision table above, and give the file an owner if you publish one.

How to tell if llms.txt is doing anything

If you publish or customize one, write down what you expect it to change before launch. "A coding agent given our docs finds the right setup page" is testable. "Our AI citations will go up" is not a supported expectation.

• Log the basics. Record the publish date, the links you included and the owner.
• Read your logs, not your hopes. A request for /llms.txt shows the file was fetched, and the user agent tells you by whom. In Ahrefs' data, most fetchers were SEO tools, so expect the same on your site.
• Don't credit it by default. If AI referral traffic moves, check everything else you changed in the same period, such as new content, fixed crawl blocks or press mentions, before giving the file credit.

The bottom line for store owners

For most e-commerce brands, llms.txt is a five-minute decision, not a project. Shopify merchants should leave the default in place. Everyone else can skip it unless they have developer docs or a large help center an agent would genuinely benefit from. The work that moves AI visibility is the unglamorous kind: letting the right crawlers in, making product and policy pages clear and consistent, and earning mentions on sites AI tools already trust.

If you want a second opinion on what to prioritize for your store's AI visibility, talk to our team or see how our AI search optimization work is scoped.