XML Sitemaps: How to Build One, Validate It, and Submit It Properly

A sitemap is not a request to be indexed. It is a machine-readable list saying these are the URLs I consider canonical and worth crawling — a discovery aid and a hint, nothing more. That distinction explains almost every sitemap disappointment: a clean sitemap speeds discovery of pages a crawler might otherwise take weeks to find, but it cannot force indexing of a page a search engine has judged thin, duplicate, or low-value.

Used correctly, though, it is one of the highest-leverage twenty minutes of work on a new site. This is the full loop: build, validate, submit, read the feedback.

What actually belongs in a sitemap

The rule is stricter than most people apply it. A URL earns a place only if it is all four of these:

• Indexable — returns HTTP 200, is not blocked by robots.txt, and carries no noindex directive.
• Canonical — it is the version you want ranked. Never list a URL whose canonical tag points somewhere else.
• Final — not a redirect. A URL that 301s to another page wastes the entry and creates a mixed signal.
• Worth a visit — a real page for a real reader.

What that excludes, and why it matters: tag and filter permutations that generate near-identical pages, internal search results, paginated duplicates of the same content, thank-you and cart pages, staging URLs, and anything behind a login. Every junk URL in a sitemap dilutes the signal that the file is a curated statement of what matters on your site.

The consistency rule is the one people miss most: your sitemap, your canonical tags, your internal links, and your robots.txt must all agree. Listing a URL in the sitemap while blocking it in robots.txt is a contradiction, and contradictions get resolved in ways you don't control.

Building it on whatever you run

You almost certainly do not need to write XML by hand.

On a CMS. WordPress ships sitemap generation in core, and the major SEO plugins replace it with a more configurable version. Shopify, Squarespace, Wix and similar platforms generate one automatically at a fixed path. On these, the work is not creating the sitemap — it is pruning it: go into the settings and exclude the archive types, tag pages, and author pages you don't want listed.

On a custom site. Generate it in your build step or from your database, so it can never drift out of date. A minimal, valid entry is just this:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/your-page</loc>
    <lastmod>2026-04-18</lastmod>
  </url>
</urlset>

<loc> is required and must be a full absolute URL, correctly escaped. <lastmod> is the only optional field worth including — but only if it is truthful. A file where every page claims to have been modified today teaches crawlers to ignore the field entirely. If you cannot populate it accurately, leave it out.

<changefreq> and <priority> are effectively ignored by major search engines. Setting every page to priority 1.0 does nothing except reveal that someone was guessing.

Static sites and one-pagers. A generator tool or a crawl-based sitemap builder is fine, provided you re-run it after every content change. A stale hand-made sitemap is worse than none.

The size limits, and the sitemap index

One sitemap file may hold up to 50,000 URLs and must stay under 50 MB uncompressed. Past either limit, split it and publish a sitemap index — a small file that lists your sitemap files rather than your pages:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap><loc>https://example.com/sitemap-pages.xml</loc></sitemap>
  <sitemap><loc>https://example.com/sitemap-blog.xml</loc></sitemap>
</sitemapindex>

Splitting by content type is worth doing well before you hit the limit, because it makes the coverage reports diagnostic: if the blog sitemap shows 90% indexed and the product sitemap shows 20%, you have just localised your problem without any extra tooling.

Validate before you submit

Five checks, in order. Each one catches a failure mode that silently wastes the submission.

1. Fetch it as a stranger. Open the sitemap URL in a private window. It must return 200 and render as XML — not a 404, not a login page, not an HTML error page with a 200 status.
2. Spot-check ten URLs. Pick ten at random, load each, and confirm: 200, no redirect, no noindex, canonical pointing at itself. If two of ten fail, assume a systematic problem, not bad luck.
3. Check for absolute, escaped URLs. Relative paths are invalid. Ampersands must be written &. One unescaped character can invalidate the whole file.
4. Confirm protocol and host match exactly. https://www. and https:// without www are different sites to a crawler. Every entry must use the host you have actually settled on.
5. Reference it in robots.txt. Add Sitemap: https://example.com/sitemap.xml on its own line. This is how crawlers that never see your submission still find the file.

Submitting it

Submit through each search engine's webmaster console — Google Search Console and Bing Webmaster Tools both have a sitemaps section where you add the path once. You submit the sitemap once; you do not resubmit after every new post. Crawlers re-fetch the file on their own schedule, which is exactly what the <lastmod> field is for.

Two supporting habits matter more than resubmission:

• The robots.txt reference, as above — it is passive, permanent, and works for engines you never manually submitted to.
• IndexNow, where your platform supports it, to ping a specific URL the moment it changes. It is complementary to a sitemap, not a replacement: the sitemap describes your whole site, the ping announces one change. The broader mechanism behind both is covered in our guide to how sites get indexed, and manual submission options are in the URL submission walkthrough.

Never submit a sitemap to bulk "submit to 500 engines" services. There are only a handful of engines that matter, they all read robots.txt, and the rest of that list is noise.

Reading what comes back

The console reports are the useful part, and each message points at a specific fix.

• "Couldn't fetch" — nearly always a 404, a redirect on the sitemap URL itself, a robots.txt block, or a firewall rule blocking the crawler's user agent. Fetch it yourself from outside your network first.
• URLs discovered but not indexed — the sitemap worked; the pages didn't persuade. This is a content and internal-linking problem, not a submission problem. Adding the URL again will not change the outcome.
• "Submitted URL marked noindex" / "blocked by robots.txt" — the contradiction described earlier. Decide which signal you meant and remove the other.
• Discovered count far below submitted count — usually a parse error partway through the file, or entries pointing at a different host.

The number to watch over time is the ratio of indexed to submitted URLs per sitemap file, not the raw total. A falling ratio while you publish more is the earliest warning that new pages are being judged as thin. How to track that alongside referral data is covered in measuring whether a submission actually worked.

FAQ

How often should I resubmit my sitemap? You don't need to. Submit once per search console, keep the Sitemap: line in robots.txt, and let the file update itself as your site changes. Resubmitting manually after each post has no additional effect.

Does a sitemap guarantee my pages get indexed? No, and any tool promising that is selling something. A sitemap makes URLs easy to discover; whether they are indexed depends on whether the search engine judges each page worth storing. Discovery and indexing are separate stages.

Should small sites bother with one? Yes, and it takes minutes on any modern platform. It matters most for brand-new sites with few inbound links, deep pages that are several clicks from the homepage, and sites where a section is poorly linked internally — exactly the situations where discovery is otherwise slow.

My sitemap lists 400 URLs but only 90 are indexed. Is it broken? Probably not. First check that the 310 are genuinely indexable and canonical — if the file is full of tag archives and filter variants, the sitemap is the problem and pruning it will help. If the excluded URLs are real content pages, the issue is page quality or internal linking, and no amount of resubmission will move it.

Sitemap clean and submitted? The next lever is getting those URLs referenced from places crawlers already visit. See the submission points and directory options at addmyurls.com and add your site where it will actually be found.