An XML sitemap is a machine-readable file that lists the URLs on your site you want search engines to crawl and index. It does not force Google to index anything, and it is not a ranking factor. What it does is give crawlers a clean, authoritative list of your important pages, along with a hint about when each one last changed. On a well-run site it quietly speeds up discovery. On a badly run site it becomes a list of everything that is wrong, handed straight to Google.
This guide covers what belongs in a sitemap, what does not, how to build and submit one, and how to use Search Console's sitemap data to find indexing problems before they cost you traffic.
Related reads: Free XML Sitemap Checker & Generator Tools
What an XML sitemap actually does
Google finds pages in two main ways: by following links, and by reading sitemaps. Links remain the primary route. A sitemap is a supplement that helps most when:
- The site is large. Thousands of URLs means some pages sit many clicks deep, and crawlers may reach them slowly.
- The site is new. With few external links pointing in, Google has fewer paths to discover your pages.
- Content changes often. News, ecommerce stock and job listings benefit from an accurate
lastmoddate that tells Google what has changed. - Some pages are poorly linked. Deep product variants, archived articles or orphan pages may only be discoverable through the sitemap. (The real fix there is better internal linking, but the sitemap is a safety net.)
For a small site with solid internal linking, Google will usually find everything anyway. A sitemap is still worth having, because Search Console's sitemap-filtered indexing report is one of the most useful diagnostics you get for free.
The basic format
A sitemap is an XML file following the sitemaps.org protocol. A minimal one looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://example.com/services/seo</loc>
<lastmod>2026-09-18</lastmod>
</url>
</urlset>Key limits and rules:
- A single sitemap file can hold up to 50,000 URLs or 50 MB uncompressed. Beyond that, split into multiple files.
- Use a sitemap index file to list several sitemaps. Splitting by content type (pages, services, blog, products) makes debugging much easier because Search Console reports on each file separately.
- URLs must be absolute and use the exact protocol and host you want indexed (https, with or without www, consistently).
- Google ignores
<priority>and<changefreq>. It does use<lastmod>, but only when it is consistently accurate.
What to include, and what to leave out
The single most important rule: a sitemap should list only canonical, indexable URLs that return a 200 status. Every URL in it is a claim that says "this page is important and I want it in the index". When those claims turn out false, Google learns to trust your sitemap less.
Include
- Pages that return HTTP 200
- Pages whose canonical tag points to themselves
- Pages without a
noindexdirective - Pages you actually want people to land on from search
Leave out
- Redirected URLs (301, 302) — list the destination instead
- 404 and 410 pages, and "soft 404" pages that show an error message with a 200 status
- Pages marked
noindex - Duplicate variants: URL parameters, sort orders, tracking tags, print versions
- Thin or placeholder pages you would not be proud to see ranking
- Pages blocked by robots.txt (listing them sends contradictory signals)
Getting lastmod right
Google has said it uses lastmod when it is "consistently and verifiably accurate". That word "consistently" matters. If your CMS updates the date on every URL whenever anything on the site changes, or you bulk-refresh dates without changing content, Google will stop trusting the field for your whole site.
Set lastmod only when the main content of the page meaningfully changes: new sections, corrected facts, updated prices. A changed footer, a new sidebar widget, or a fixed typo does not count.
How to create a sitemap
WordPress: Core WordPress generates a basic sitemap at /wp-sitemap.xml. Most sites use Yoast SEO or Rank Math instead, which produce a sitemap index at /sitemap_index.xml and let you exclude post types, taxonomies and individual pages.
Shopify, Wix, Squarespace: A sitemap is generated automatically at /sitemap.xml. You cannot edit it directly, so control what appears in it by hiding or noindexing pages.
Custom or headless sites: Generate the sitemap from your database or CMS at build time or on request. Pull only published, indexable records and use each record's real content-updated timestamp for lastmod. Next.js, Nuxt and Astro all have first-party or well-maintained sitemap tooling.
Crawler-based generators: Tools like Screaming Frog can crawl your site and export a sitemap. This works for static sites, but the file goes stale the moment content changes, so treat it as a one-off rather than a long-term setup.
Submitting your sitemap
- Reference it in robots.txt. Add a line such as
Sitemap: https://example.com/sitemap.xml. Every major search engine reads this. - Submit it in Google Search Console. Go to Indexing → Sitemaps, enter the URL and submit. If you use a sitemap index, submitting the index is enough.
- Submit it in Bing Webmaster Tools. Bing's index also feeds several AI assistants, so it is worth the two minutes.
You do not need to resubmit every time the file changes. Google refetches known sitemaps on its own schedule.
Using sitemap data to find indexing problems
This is where a sitemap earns its keep. In Search Console, open Indexing → Pages and use the filter at the top to switch from "All known pages" to "All submitted pages", or to a specific sitemap file. You now see indexing status only for the URLs you said matter.
Look for these patterns:
- Submitted pages in "Not found (404)" or "Page with redirect". Your sitemap is out of date. Fix the generator so it only outputs live, final URLs.
- Submitted pages in "Alternate page with proper canonical tag". Your sitemap lists URLs that point their canonical elsewhere. List the canonical version instead.
- A high share in "Crawled – currently not indexed". Google fetched the pages and decided they were not worth indexing. This is almost always a content quality or duplication issue, not a technical one.
- A high share in "Discovered – currently not indexed". Google knows the URLs exist but has not crawled them yet. On small sites this usually signals that Google does not yet rate the site highly enough to crawl everything promptly.
A healthy site typically sees the large majority of submitted URLs indexed. If a big slice of your sitemap is being crawled but not indexed, adding more URLs will not help. Improving or consolidating the existing ones will.
Common sitemap mistakes
- Listing everything. Dumping every URL the CMS knows about, including tags, author archives and parameter variants, dilutes the signal.
- Stale files. A sitemap built once and never updated slowly fills with 404s and redirects.
- Mismatched hosts. Listing http URLs on an https site, or www URLs when the canonical host is non-www.
- Fake freshness. Updating
lastmodon hundreds of URLs at once without real content changes. - Blocking the sitemap. Serving it behind authentication, a firewall rule, or a robots.txt disallow.
A quick sitemap health check
- Open your sitemap in a browser and confirm it loads as valid XML.
- Crawl every URL in it (Screaming Frog's list mode works well) and confirm each returns 200, is self-canonical and is not noindexed.
- In Search Console, check the Sitemaps report shows "Success" and a recent read date.
- Filter the Pages report by your sitemap and review every "not indexed" reason.
- Spot-check
lastmodvalues against when content really changed.
If you would rather have this handled as part of a full technical review, our SEO services team audits sitemaps, crawlability and indexing together. For the bigger picture, see our step-by-step technical SEO audit guide.
