XML Sitemap
An XML sitemap lists the URLs you want crawlers to find, with last-modified dates. Learn how to build one and what it does and does not guarantee.
TL;DR
An XML sitemap is a file listing the URLs on your site that you want crawlers to discover, each optionally carrying the date it last changed. It is a discovery aid rather than an instruction: crawlers use it to find pages, then decide independently whether to index them. For AI visibility its main contribution is the lastmod date, since engines that weight freshness need a signal about what changed and when.
Why XML sitemaps matter
Crawlers find most pages by following links. A sitemap covers the cases where linking alone is unreliable.
- Discovery: Pages buried deep in your architecture, or reachable only through search and filters, get found.
- Freshness signalling: Accurate lastmod values tell crawlers which pages are worth revisiting.
- Large site coverage: Beyond a few thousand URLs, link-following alone leaves gaps that a sitemap closes.
- Diagnostics: Comparing submitted URLs against indexed URLs in Search Console shows exactly where indexing breaks down.
How to build an XML sitemap
Include only canonical, indexable URLs
Every entry should be a page you want in the index: 200 status, self-canonical, not noindexed.
Set lastmod honestly
Update it when content genuinely changes. Bumping every date nightly teaches crawlers to ignore the field entirely.
Split large sitemaps
The limits are 50,000 URLs or 50MB uncompressed. Beyond either, split into multiple files behind a sitemap index.
Segment by content type
Separate sitemaps for products, articles and documentation make Search Console diagnostics far more readable.
Reference it in robots.txt
A Sitemap line at the root file makes it discoverable without manual submission.
Generate it automatically
A sitemap maintained by hand goes stale within a release cycle. Build it from your content source.
XML sitemap vs. HTML sitemap
XML sitemap: Machine-readable, submitted to search engines, carrying metadata like lastmod. Users never see it.
HTML sitemap: A human-readable page listing site sections, which also creates real internal links crawlers can follow.
The XML version aids discovery; the HTML version aids navigation and passes link signals. Neither replaces good internal linking, which remains the primary way both users and crawlers move through a site.
Submitting URLs is not the same as having them fetched, indexed or cited. Findrix checks sitemap health and crawler access in a 31-point technical audit, then measures whether seven AI engines actually cite your pages against named competitors. Every gap comes with the fix already written: technical, content and off-site. The audit is free, takes about a minute, and requires no signup.
What belongs in a sitemap
- Canonical URLs only: Never include a page that canonicals elsewhere.
- Indexable pages: Anything carrying noindex should be excluded.
- Live URLs: Redirects and 404s in a sitemap waste crawl attention and signal neglect.
- Accurate lastmod: Real modification dates, in ISO 8601 format.
- Segmented files: Grouped by type so coverage problems are traceable.
What a sitemap cannot do
It cannot force indexing. Submitting a URL is a suggestion, and engines routinely decline pages they judge thin, duplicative or unimportant. Teams treating sitemap submission as a fix for indexing problems are usually looking at a content quality issue wearing a technical costume.
It also cannot compensate for weak internal linking. A page reachable only through the sitemap is a page nothing on your site considers worth linking to, and crawlers read that absence as a signal about importance.
The commonest failure is neglect. A sitemap full of redirects and dead URLs teaches crawlers to distrust the whole file.
Frequently asked questions
Do I need an XML sitemap?
Small, well-linked sites often manage without one. Anything large, frequently updated, or with pages that are hard to reach through navigation benefits materially. The cost is near zero, so the default should be yes.
Does a sitemap guarantee indexing?
No. It helps crawlers discover URLs. Whether a page gets indexed depends on quality, duplication and how the engine judges its value. Submission and indexing are separate decisions.
Should I include noindexed pages in my sitemap?
No. A sitemap is a list of pages you want indexed, so including noindexed URLs sends contradictory signals and clutters your coverage reports with expected failures.