New: The $100M AI Visibility Roadmap, built from 7,960 live pages and 197 A/B tests. Get it free →

Home / Blog / The Future of SEO: How AI Is Rewriting the Rules

AI Search · 12 min read

The Future of SEO: How AI Is Rewriting the Rules

AI search still runs on crawling, indexing and ranking, but it now splits one question into many searches, reads your product feed and cites few sources. Here are the rules that changed and how to adapt.

Robert Battle · Updated

Key takeaways
  • Google says its generative AI features are rooted in its core ranking and quality systems, so SEO fundamentals still decide who gets cited.
  • Ahrefs found 37.9% of AI Overview citations rank in Google's top 10 for the same query, so answering sub-questions matters as much as one ranking.
  • Cloudflare measured OpenAI crawling about 1,091 pages per referral in July 2025, versus about 5 for Google, so treat each AI crawler as a separate decision.
  • Google's Shopping Graph holds over 50 billion product listings, so your product feed is now an SEO asset that AI answers quote directly.
  • Start by unblocking search crawlers, fixing your feed and answering the top buyer questions on your 20 highest-revenue pages.

The future of SEO is being decided by how AI systems pick their sources. Instead of matching one query to one ranked list, Google AI Mode, AI Overviews, ChatGPT and Perplexity break a question into many searches, pull passages from pages, product feeds and third-party sites, and cite only a few, so the rules shift from "rank one page for one keyword" to "be the most useful, crawlable and trusted source across every version of the question."

This guide is for e-commerce and Shopify teams who want to understand the mechanics behind that change, not just the headlines. You'll see which old SEO rules still hold, which ones AI has rewritten, what the latest data says about how AI answers choose citations and how much AI crawlers take versus send back, why product data is turning into its own ranking surface, and a step-by-step plan to adapt. If you want the market view (adoption numbers, click loss and agency budgets), read our companion piece on the future of the SEO industry.

What is the future of SEO?

For two decades, SEO followed a simple loop: a person types a query, a search engine ranks ten links, and the best-ranked page earns the click. AI search keeps the index and the ranking systems but adds a layer on top. A language model reads the question, decides what it needs to know, runs its own searches, reads the results and writes an answer with a handful of links.

Google is direct about where that leaves site owners. Its guide to optimizing for generative AI features (last updated July 10, 2026) says these features are "rooted in our core Search ranking and quality systems." It tells you to keep pages crawlable and indexable, to publish "non-commodity" content with a unique viewpoint, and to skip shortcuts such as chunking content, rewriting text only for AI or chasing inauthentic mentions.

So the future of SEO is not a new discipline with new tricks. It is the same discipline with three rewritten rules:

  • The unit of competition changes. You compete for a passage inside an answer, across many related sub-queries, not only for a position on one results page.
  • The audience changes. Machines (crawlers, retrieval systems, shopping agents) read your site before people do, and they read your product feed as closely as your pages.
  • The proof changes. What other sites, reviewers and publications say about you counts more, because AI answers lean on third-party sources to decide who to trust.

Most of what works in SEO still works. The difference is where the effort pays off. Use this table to see which habits to keep, which to adjust and which to drop.

RuleHow it workedHow AI rewrites itWhat to do now
Target one keyword per pageA page ranked for a head term and its close variantsAI expands one question into many related searches and blends the resultsCover the full set of follow-up questions a buyer asks, on the right page
Rank in the top 10Top positions earned most clicksAI answers cite many pages from outside the top 10Keep ranking, and also make each section quotable on its own
Write for the searcherA person scanned the pageA model extracts a passage, then a person decides whether to clickLead every section with a direct answer, then add depth
Earn backlinksLinks were the main off-site signalBrand mentions and third-party coverage carry more weight in who gets namedEarn reviews, press and expert mentions, not just links
Optimize the product pageTitle, description and schema on the pageShopping surfaces read feeds, prices and stock in near real timeTreat your product feed as an SEO asset
Allow GooglebotOne crawler mattered mostSeveral AI crawlers each control a different surfaceDecide crawler by crawler, based on what each one does

Notice what is missing from the right-hand column: special AI files, hidden prompts or content written only for bots. Google says it does not use llms.txt or "special" AI markup, and that no special schema is needed for AI features. The new rules reward the same quality signals, applied more widely.

How AI search engines choose their sources

To work with AI search, it helps to know what happens between the question and the answer. Google calls the key step query fan-out: "a set of concurrent, related queries generated by the model." Its example is a question about fixing a weedy lawn, which branches into searches about herbicides and weed prevention. The model gathers results for each branch and writes one answer.

What fan-out means for a store

Picture a shopper asking AI Mode for "the best running shoes for flat feet under $150." The model may search for stability shoe types, arch support, specific models, price ranges and recent reviews. A product page that only says "Lightweight running shoe, free shipping" matches none of those branches. A page that states the support level, the drop, who it suits, the price and what reviewers say matches several.

That is the practical meaning of "cover the topic." It is not about word count. It is about answering the sub-questions a model is likely to ask on the pages where those answers belong: product pages for specs and fit, collection pages for comparisons, guides for the "which one should I buy" questions.

Ranking still matters, but it is no longer the whole story

Ahrefs measured how often pages cited in AI Overviews also rank in Google's top 10 for the same query. In its March 2026 update, covering 863,000 keyword results and 4 million AI Overview URLs, 37.9% of cited URLs appeared in the top 10. The rest split almost evenly between positions 11 to 100 and pages outside the top 100.

Where AI Overview citations rank in GoogleShare of cited URLs by Google position for the same query; 863K keyword SERPs, 4M AI Overview URLs
37.9% Top 10 31.2% 11 to 100 31% Beyond 100
View the data
Value
Top 1037.9%
11 to 10031.2%
Beyond 10031%

Source: Ahrefs, Update: 38% of AI Overview Citations Pull From The Top 10, March 2026

In Ahrefs' previous version of the study, from July 2025, about 76% of citations came from the top 10. Ahrefs notes two reasons the comparison isn't perfect: AI Overviews moved to Gemini 3 in January 2026, and its own parsing improved so it now sees more citations. Even with those caveats, the direction is useful. Fan-out pulls in pages that rank for the sub-queries, not only the main query.

The takeaway is not "stop caring about rankings." Pages in the top 10 are still the single largest group of citations. The takeaway is that a page ranking 14th for your head term, but first for a specific sub-question, can still be the source the answer quotes.

See how AI search reads your store today

We benchmark your store against your top three competitors in Google and AI answers, flag the pages ready to move and show the prioritized fixes with the revenue math.

Get my AI visibility report →

Why AI crawlers take more than they send back

The second rewritten rule is about the bargain between sites and search engines. Classic search crawled your pages and sent visitors in return. AI platforms crawl far more per visit they refer, and much of that crawling feeds model training rather than search.

Cloudflare tracks this with a crawl-to-refer ratio: HTML pages a platform's crawlers request for every visit it refers back. In its August 2025 analysis, the July 2025 ratios looked like this.

Pages crawled for every visit referred, July 2025Crawl-to-refer ratio; Anthropic (about 38,066) is left off so the scale stays readable
OpenAI1,091.4
Perplexity194.8
Microsoft40.7
Google5.4
View the data
Value
OpenAI1,091.4
Perplexity194.8
Microsoft40.7
Google5.4

Source: Cloudflare, The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals, August 2025

Anthropic was off the scale at roughly 38,066 crawls per referral in July 2025, down from about 286,930 in January. Cloudflare notes that Claude's native app doesn't send a referrer header, so some referrals go uncounted and the ratios may be overstated. Google, by contrast, crawled about 5 pages per referral.

Why the gap? Most AI crawling isn't for search at all. Cloudflare's July 2025 data split AI crawler traffic by purpose.

Why AI crawlers visit sites, July 2025Share of AI crawler traffic by purpose across Cloudflare's network
79% Training 17% Search 3.2% User actions
View the data
Value
Training79%
Search17%
User actions3.2%

Source: Cloudflare, The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals, August 2025

What this means for your robots.txt

Because different bots do different jobs, the future of technical SEO is crawler-by-crawler decisions instead of one blanket rule. Blocking a training bot doesn't have to remove you from AI search, but blocking a search bot will.

Crawler or tokenOperatorWhat it doesIf you block it
GooglebotGoogleCrawls for Google Search, including AI Overviews and AI ModeYou drop out of Google Search and its AI features
Google-ExtendedGoogleA robots.txt token that controls use of your content for Gemini training and groundingNo effect on Google Search inclusion or rankings, per Google
OAI-SearchBotOpenAIIndexes pages for ChatGPT search resultsYour pages can't appear as ChatGPT search sources
GPTBotOpenAICollects content that may train OpenAI modelsNo effect on ChatGPT search, per OpenAI
PerplexityBotPerplexityIndexes pages for Perplexity answers and linksYour pages are less likely to be cited in Perplexity

For most stores, the sensible default is to allow the search crawlers and make a deliberate business decision on the training ones. Our guide to GPTBot vs OAI-SearchBot covers the OpenAI side in detail, and the free robots.txt tester shows what your current file allows.

Product data is becoming its own ranking surface

For e-commerce, the biggest rule change is that your product data now travels without your website. AI shopping answers are built from structured product information: titles, prices, availability, variants, reviews and shipping. Google says AI Mode shopping draws on its Shopping Graph, and at the National Retail Federation in January 2026, Sundar Pichai gave its scale.

The product data behind AI shopping answers
50B+Product listings in Google's Shopping Graphincluding inventory, prices and reviews
2B+Listings refreshed every hourso stale feed data gets noticed fast

Source: Google, Sundar Pichai's remarks at the 2026 National Retail Federation, January 2026

A catalog that large, refreshed that often, means AI answers can quote your current price and stock level. It also means a stale feed, a missing size or a wrong price gets quoted just as confidently.

Checkout is moving into the answer

In the same remarks, Google introduced the Universal Commerce Protocol (UCP), built with Shopify, Etsy, Wayfair, Target and Walmart. Google's Merchant Center help page describes UCP-powered checkout as a "Buy" button on eligible product listings in AI Mode and Gemini, for select merchants in the United States, Canada and Australia, with the retailer remaining the seller of record. Shopify's agentic storefronts help page describes a similar path for eligible Shopify stores into AI channels.

When a shopper can compare, choose and buy inside an AI answer, the product feed effectively becomes a landing page. Google's AI guide makes the same point in plainer terms: Merchant Center feeds can help products "be visible in both AI responses and other Google Search results."

What to fix first in your catalog

  • Titles that describe, not just name. "Women's Merino Running Tee, Moisture Wicking, Black" beats "The Stride Tee."
  • Complete attributes. Size, color, material, gender, GTIN and condition, filled for every variant.
  • Prices and stock that match the page. Mismatches can get products disapproved and erode trust in the listing.
  • Product structured data on the page. Not required for AI features, but it supports rich results and keeps page and feed consistent. Check it with the schema validator and our guide to product schema on Shopify.

What AI can't replace: content only you can write

If AI can summarize everything already on the web, a page that only summarizes the web has nothing left to offer. That is the logic behind Google's push for "non-commodity" content: original data, first-hand experience and a clear point of view.

Google's helpful content guidance asks whether content provides "original information, reporting, research, or analysis" and whether it offers "insightful analysis or interesting information that is beyond the obvious." Its guidance on high-quality reviews asks for evidence of real experience, quantitative measurements and honest pros and cons.

Stores are well placed to deliver this, because they hold information nobody else has:

  • Fit and sizing data from returns and exchanges ("runs half a size small" backed by what customers actually did).
  • Use-case testing from your own team: photos, measurements, side-by-side comparisons.
  • Customer questions from support tickets and reviews, answered on the product page.
  • Expert voices such as founders, formulators, designers or in-house specialists, named on the page with real credentials.

This is also where brand authority comes from. AI answers lean on what others say about you, so original content that gets cited, reviewed and discussed elsewhere compounds. Our guide to thought leadership SEO covers how to build that on purpose.

The new rules of search, in one roadmap

Our free $100M AI Visibility Roadmap, built from 7,960 live pages and 197 A/B tests, shows how to get cited by ChatGPT, Google AI Overviews and Perplexity.

Get the free roadmap →

Here is the order we'd work in for a Shopify or e-commerce store. Each step builds on the one before, so resist skipping to the shiny parts.

  1. Confirm machines can reach you. Check robots.txt, your CDN or bot-protection settings and your sitemap. Make sure Googlebot, OAI-SearchBot and PerplexityBot can fetch your product, collection and guide pages. Run the sitemap checker and the robots.txt tester.
  2. Fix your product feed. Bring titles, attributes, prices and stock in line with your pages in Merchant Center. This now affects AI shopping answers as well as Shopping ads and free listings.
  3. Map the fan-out for your top products. For each of your 20 highest-revenue products or collections, list the 10 to 15 questions a buyer asks before purchase. Pull them from support tickets, reviews, on-site search and the "People also ask" box.
  4. Answer those questions where they belong. Specs and fit on product pages, comparisons on collection pages, "which should I buy" on guides. Start each section with a one or two sentence answer.
  5. Add what only you know. Original photos, measurements, test results and named expert input. Remove generic copy that could sit on any competitor's site.
  6. Earn third-party proof. Reviews, gift guides, niche publications, creator coverage and expert roundups. Fix unlinked mentions and inconsistent brand facts across the web.
  7. Measure the new surfaces. Track AI referrals in GA4, Google's generative AI reporting in Search Console where you have it, and a fixed set of buyer prompts you test monthly. Our AEO checker grades how answer-ready a page is.

Most stores can finish steps 1 and 2 in a couple of weeks. Steps 3 to 6 are ongoing work, best done in monthly sprints on the pages that drive revenue.

Signals to watch over the next 12 months

The specifics will keep moving. These are the signals that tell you whether the rules are shifting further, and what each one should change in your plan.

SignalWhy it mattersWhat to do if it grows
Checkout inside AI answersShoppers may buy without visiting your sitePrioritize feed quality, return policy clarity and review volume
AI citations from outside the top 10Sub-query coverage matters more than one rankingExpand answer coverage on existing pages
Crawl-to-refer ratiosShows which platforms send value backRevisit which training crawlers you allow
AI referral traffic in GA4Your own measure of what AI sendsBuild landing pages that match how AI visitors arrive
Brand mentions in AI answersShows whether models name you unpromptedInvest in reviews, press and expert content

Track these quarterly. If a signal hasn't moved, don't rebuild your strategy around a headline.

Where to start this week

The future of SEO rewards the same work it always has, aimed at more readers. Open your robots.txt and confirm the search crawlers are allowed. Pull your top 20 revenue pages and compare their product data with Merchant Center. Then pick five of those pages and answer the three questions buyers ask most, in the first lines of the relevant section, with something only your brand could say.

If you'd like a second pair of eyes on where your store stands in Google and AI answers today, we're happy to take a look.

Ready to adapt your SEO for AI search?

Book a free 30-minute call. We'll compare you with your top three competitors, show which pages to fix first and estimate what it's worth.

Get my custom plan →
FAQ

Questions & answers

Is SEO still worth investing in with AI search?
Yes. Google says its AI features are built on its core ranking and quality systems, and AI answers cite pages they can crawl and trust. What changes is how you measure the return: citations, AI referrals and branded search alongside organic revenue.
What is query fan-out in SEO?
Query fan-out is when an AI search system splits one question into several related searches, gathers results for each and writes one answer. It means pages that answer specific sub-questions well can be cited even if they don't rank top 10 for the main query.
Should I block AI crawlers on my Shopify store?
Block or allow each crawler based on what it does. Search crawlers such as Googlebot, OAI-SearchBot and PerplexityBot decide whether you can appear in those answers, while training crawlers such as GPTBot and the Google-Extended token don't affect search visibility, according to OpenAI and Google.
Do I need llms.txt or special schema for AI search?
Not for Google. Google says it doesn't use llms.txt or special AI markup, and no special schema is needed for its AI features. Standard structured data is still worth keeping for rich results.
How does AI search change product page SEO?
AI shopping answers read structured product data such as prices, stock, variants and reviews, often from your feed as well as your page. Accurate feeds, descriptive titles and pages that answer fit and comparison questions matter more than before.
Free custom plan

Find the revenue you're leaving on the table

A free 30-minute call where we build the first version of your plan with you. You leave with a plan either way.

  • Your score across Google, AI citations, branded demand, and reviews
  • The pages on your site that are ready to move right now
  • Your first Flagship page and the first three tests we'd run

Step 1 of 3

Get your free custom plan

Free. Takes about 30 seconds.