XML Sitemap Best Practices: What to Include and Exclude

XML sitemap best practices come down to one rule: list only the canonical, indexable URLs you want in search results, with accurate lastmod dates, and keep every file under Google's limit of 50,000 URLs or 50 MB uncompressed. Then submit it in Search Console, reference it in robots.txt, and check it every month so redirected, deleted and noindexed URLs never pile up.

This guide is for e-commerce and Shopify teams who want their sitemap to help crawling instead of just existing. You'll get the current limits and format rules from Google and sitemaps.org, which tags Google and Bing actually read, an include and exclude table, how Shopify's automatic sitemap behaves, a step-by-step submission and monitoring routine, and a fix list for the errors Search Console reports.

What is an XML sitemap, and do you need one?

An XML sitemap is a file that lists the URLs on your site you want search engines to crawl, with optional details such as when each page last changed. It helps Google and Bing discover new pages and decide which known pages to recrawl. It does not force anything: Google's documentation says submitting a sitemap is "merely a hint" and doesn't guarantee that every URL will be crawled or indexed.

Google says you may not need a sitemap if your site is small, which it defines as about 500 pages or fewer that you want in search results, and every important page is reachable through internal links. You likely do need one if any of these are true:

• Your site is large. Most stores pass 500 indexable URLs quickly once you count products, collections, blog posts and pages.
• Your site is new or has few external links. Crawlers may not find pages nobody links to.
• You have a lot of images or video you want to appear in search.
• Your catalog changes often. New products, restocks and seasonal collections need to be found and recrawled quickly.

For almost every e-commerce store, the answer is yes. The real question is whether your sitemap sends a clean signal or a noisy one.

XML sitemap limits and format rules

The rules below come from Google Search Central and the sitemaps.org protocol, which Google, Bing and other engines follow. Break one and Search Console will usually flag the file as having errors, or fail to read it entirely.

Size and count limits

• 50,000 URLs or 50 MB per sitemap, measured uncompressed. Whichever you hit first, split the file. You can gzip a sitemap to save bandwidth, but the 50 MB limit still applies to the uncompressed size.
• Sitemap index files group multiple sitemaps. One index can list up to 50,000 sitemaps, and Google lets you submit up to 500 sitemap index files per site in Search Console.
• No nesting. A sitemap index can list sitemaps, not other sitemap indexes. Search Console reports "Nested sitemap indexes" as an error.
• Image sitemaps allow up to 1,000 <image:image> entries per URL.

Bing published the same 50,000 URL limit in July 2025 and noted that one index file can therefore point to 2.5 billion URLs. Few stores get anywhere near that, but large catalogs with variants, multiple markets and blogs can outgrow a single file faster than you'd expect.

Format rules

• UTF-8 encoding for the whole file.
• Absolute URLs only. Use https://www.example.com/products/blue-mug, never /products/blue-mug. Google crawls URLs exactly as listed, so the protocol, host and trailing slash must match your canonical URLs.
• Same host and protocol as the sitemap. A sitemap on https://www.example.com should not list http://example.com URLs. Search Console calls this a path mismatch.
• Location sets scope. Per sitemaps.org, a sitemap at /catalog/sitemap.xml can only list URLs under /catalog/. Put your sitemap or index at the root so it can cover everything.
• Escape special characters. An ampersand in a URL must be written as &, and the same goes for quotes and angle brackets. An unescaped & is one of the most common causes of a parsing error.
• URLs under 2,048 characters, per the sitemaps.org protocol.

A minimal valid entry looks like this: <url><loc>https://www.example.com/products/blue-mug</loc><lastmod>2026-10-20</lastmod></url>, inside a <urlset> root element that declares the sitemaps.org namespace.

Which sitemap tags Google and Bing actually use

The protocol defines four tags for each URL. Only two matter in 2026, and one of them only matters if you keep it honest.

Get lastmod right

When Google retired the sitemap ping endpoint in June 2023, it also explained how it treats lastmod: it uses the value as a signal for scheduling recrawls of known URLs, but only if it matches reality. If a page changed years ago and your sitemap says it changed yesterday, Google said that eventually "we're not going to believe you anymore." Google counts a change as significant when you edit the main content, change structured data or update links. A new copyright year in the footer is not a reason to update lastmod.

Bing's July 2025 guidance says the same thing from the other side: lastmod helps Bing decide what to recrawl and what to skip, and it should reflect when the page content changed, not when the sitemap file was generated. Bing recommends full ISO 8601 timestamps with date and time.

Google also notes it's fine to leave lastmod out for pages where your software can't tell when they changed, such as a homepage or category page that just aggregates other pages. A missing date is better than a wrong one.

What to include in your XML sitemap

Your sitemap should be a list of pages you would be happy to see as a search result landing page. For a store, that means every URL that is canonical, returns a 200 status, is not blocked by robots.txt and has no noindex tag. Google's own guidance is to list the canonical URLs you want shown in results.

• Product pages at their canonical URL, the one your rel="canonical" points to.
• Collection or category pages you want to rank, at their base URL without sort or filter parameters.
• Blog posts and guides that are live and indexable.
• Core content pages such as About, size guides, shipping and returns pages that answer real searches.
• Indexable filtered landing pages, only if you've deliberately built them as static, unique pages with their own canonical. Our guide to faceted navigation SEO covers when that's worth doing.

Two optional additions help stores in particular. Image entries help Google find product images, and Google now only documents <image:loc> (it removed caption, title, geo location and license from its docs). For international stores, you can declare hreflang alternates in the sitemap with xhtml:link elements, and each URL must list every language version, including itself.

If you don't have a sitemap yet, or need one for a non-Shopify site, our free XML sitemap generator crawls a small site (up to 300 pages) or turns a pasted URL list into a valid file.

What to exclude from your sitemap

Every URL in your sitemap is a claim that the page deserves crawling and indexing. Listing pages you've told search engines to ignore sends a mixed signal and wastes the attention you want on products and collections. This is where most store sitemaps go wrong.

Out-of-stock and discontinued products

Temporarily out-of-stock products can stay in the sitemap if the page is still live and you expect to restock, since the URL keeps its history. Permanently discontinued products are different. Either redirect them to the closest replacement or collection and remove the old URL from the sitemap, or let them return a 404 or 410. Google's crawl budget guide also warns that soft 404s, pages that say "not found" but return a 200, keep getting crawled and waste budget.

Google's crawl budget guidance is aimed at large sites (roughly 1 million+ pages) and medium sites with 10,000+ pages that change daily, and it calls those numbers rough estimates. Smaller stores rarely have a true crawl budget problem, but a clean sitemap still speeds up discovery and makes Search Console reports far easier to read. Our guide to crawl budget optimization goes deeper.

How Shopify sitemaps work

Shopify generates your sitemap automatically at yourdomain.com/sitemap.xml. According to Shopify's help center, it links to separate child sitemaps for products, collections, blogs and pages, includes each product's primary image, and updates automatically when you add a product, collection, page, image or blog post. On a live Shopify store we checked on October 27, 2026, the generated file itself says it "can not be edited manually."

On that same store, the product sitemap set <changefreq>daily on every URL and included image title and caption tags. Both are harmless; Google ignores changefreq and no longer documents those image tags. Larger stores also get numbered files, such as sitemap_collections_2.xml, once a type is split.

What Shopify store owners should actually do

• Use the right hiding method. Shopify's help center lists the seo.hidden metafield (for products, pages and blog posts) and unlisted status (for products) as the ways to keep a resource out of the sitemap. A noindex tag added in your theme hides the page from search engines but isn't listed as removing it from the sitemap, so check the file after you add one.
• Turn off password protection before launch. Shopify notes that a sitemap reader can't access the file while the store is in private mode.
• Clean up at the source. Because you can't edit the file, the fix for a messy sitemap is to delete, redirect, merge or hide the underlying products, collections and pages.
• Leave filtered collection URLs out of the index. Shopify's sitemap lists collections at their base URL. Sorted and filtered versions should canonicalize to that base and, on the store we checked, are blocked in robots.txt. Our Shopify SEO checklist covers the rest.

How to submit and monitor your sitemap

Google retired its sitemap ping endpoint (the deprecation was announced in June 2023, and requests to it now return a 404), so pinging no longer does anything. Use Search Console, robots.txt and, for Bing, Bing Webmaster Tools and IndexNow. Here is the routine to set up once and repeat monthly.

1. Validate the file. Run your sitemap or sitemap index through our free XML sitemap checker. It validates the XML and spot-checks that listed URLs return 200 and are indexable.
2. Reference it in robots.txt. Add a line such as Sitemap: https://www.example.com/sitemap.xml with the full URL. Google allows multiple Sitemap lines. Shopify adds this line for you. Confirm with our robots.txt tester that the sitemap itself isn't blocked.
3. Submit it in Search Console. Open Indexing, then Sitemaps, enter the sitemap or index URL and submit. You need owner permissions on the property. For multiple domains or markets, submit each one.
4. Submit it in Bing Webmaster Tools. Bing says it fetches a submitted sitemap immediately and then revisits it at least once a day. Bing also recommends IndexNow for real-time change notifications alongside sitemaps.
5. Check the status. In the Sitemaps report, a sitemap shows Success, Has errors or Couldn't fetch. "Has errors" still queues the URLs Google could parse, but fix the errors anyway.
6. Compare submitted and indexed pages. In the Page indexing report, filter to submitted pages. Large groups under "Excluded by noindex," "Page with redirect" or "Alternate page with proper canonical tag" mean your sitemap lists URLs it shouldn't.
7. Recheck after big changes. Migrations, theme changes, catalog cleanups and new markets are when sitemaps break. Run the checker and the redirect checker on old URLs the same day.

The submitted pages comparison is the most useful check in the list. A healthy store sitemap should have very few submitted URLs that end up excluded for reasons you control, such as redirects, noindex tags or canonicals pointing elsewhere.

Common sitemap errors and how to fix them

Google's Sitemaps report help page lists the errors below, among others. Most come down to a URL that doesn't match your canonical, a server problem or malformed XML.

Google also says that if a sitemap fetch keeps failing, it retries for a few days and then stops, so don't leave a "Couldn't fetch" status sitting for weeks. Temporary errors usually resolve on their own without resubmitting.

Problems Search Console won't flag as errors

• Stale URLs that still return 200. Old campaign pages, empty collections and thin pages can be perfectly valid XML and still not belong in your sitemap.
• Canonical mismatches. If a listed URL canonicalizes to a different URL, it shows in the Page indexing report, not as a sitemap error. Our guide to canonical tags explains how to align them.
• Important pages missing from the sitemap. Compare your sitemap to a crawl of your site and your top landing pages. A revenue page missing from both your sitemap and your internal links is hard to find.

Your XML sitemap checklist for this week

If you only have an hour, work through this list in order. It catches most of the sitemap problems we see on e-commerce sites.

• Open /sitemap.xml and confirm it loads, lists child sitemaps and uses your canonical host and https.
• Run it through the sitemap checker and fix any non-200, redirected or noindexed URLs at the source.
• Confirm robots.txt has a Sitemap line and doesn't block the sitemap.
• Check that the sitemap is submitted in Search Console and Bing Webmaster Tools, with a Success status.
• In the Page indexing report, filter to submitted pages and investigate the largest excluded groups.
• Spot-check five lastmod values against what actually changed on the page.
• On Shopify, hide thin pages and retired products with the seo.hidden metafield or unlisted status, and redirect discontinued products to their closest replacement.

A sitemap won't rank a weak page, but a clean one makes sure your best products and collections are found, recrawled quickly when they change, and easy to diagnose when something breaks. If you'd like a second set of eyes on your sitemap, indexing and the rest of your technical SEO, our technical SEO team can help, or you can book a free strategy call.