5 Docusaurus Sitemap Checks Before a Docs Release

A Docusaurus sitemap is ready for release when it lists the exact public URLs you intend to expose, leaves out pages that should stay out of search, and reaches visitors as valid XML from the deployed build. A successful build or an unchanged URL count does not establish all three.

These five checks follow a documentation release from configuration to the public response. They include a reusable script that compares sitemap URLs with an approved release inventory, including missing, unexpected, and duplicate entries. Examples use the native sitemap plugin documented for Docusaurus 3.10.2; check your installed package versions before applying them to an older project or a custom plugin.

For titles, page structure, and the wider documentation audit, use our Docusaurus SEO checklist. Here, the goal is narrower: prove what your sitemap contains and what the deployed site actually serves. A correct sitemap helps discovery; it does not guarantee that Google will index every URL.

1. Build with the correct public URL settings

Inspect the sitemap from a production build, not the development server. The native sitemap plugin is inactive during development, and @docusaurus/preset-classic already includes it. If that preset owns your sitemap configuration, edit its sitemap options rather than registering a second copy of the plugin.

From your Docusaurus project directory, check installed versions and run the project's production build command:

npm ls @docusaurus/core @docusaurus/plugin-sitemap
npm run build

Docusaurus normally writes production files to build/. If your build command changes the output directory, inspect that directory instead. Confirm the sitemap belongs to this release, rather than accepting an old XML file left in an upload folder.

For a hypothetical site published at https://docs.example.com/help/, the relevant settings could look like this:

// Relevant excerpt from docusaurus.config.js
export default {
  url: 'https://docs.example.com',
  baseUrl: '/help/',
  trailingSlash: true,
  presets: [
    ['classic', {
      sitemap: {
        filename: 'sitemap.xml',
        lastmod: null,
      },
    }],
  ],
};

The origin belongs in url; the hosting path belongs in baseUrl. In this example, a docs route might produce https://docs.example.com/help/docs/install/. It should not acquire a second /help/, use a preview hostname, or lose its path prefix.

trailingSlash: true is an example choice, not a requirement for every host. The site configuration reference explains how each setting changes URLs and emitted HTML. Choose the URL form your host actually serves, then compare sitemap entries, redirects, internal links, and canonical tags against it.

Keep the build as your reference artifact once it produces the intended file and representative <loc> values use the public origin, path prefix, and URL form.

2. Reconcile every expected release URL

Create an inventory independently of the sitemap. Copying <loc> values into a manifest and comparing the file with itself proves nothing. Start with the release's supported versions, docs plugin instances, locale configuration, approved routes, and deliberate search exclusions.

Be careful with Docusaurus terminology: the current source version can be the next, unreleased documentation, while the latest frozen version uses the main docs path. Read versions.json, lastVersion, includeCurrentVersion, onlyIncludeVersions, and any path overrides before deciding which URLs should exist. The versioning guide shows why a folder name alone is insufficient.

Use a small inventory like this before expanding it into exact URLs. These rows illustrate decisions, not mandatory route patterns:

Docusaurus locales can be built independently. A root sitemap that lacks another language's URLs does not by itself prove those pages were omitted: inspect that locale's own output. Conversely, a working locale homepage does not prove its sitemap was deployed. This maintainer discussion about missing localized URLs illustrates the question; the current i18n architecture explains the separate builds.

Compare exact sets, not just totals

Save every eligible URL for the scope you are checking in expected-urls.txt, one absolute URL per line. Include legitimate non-docs routes if the XML covers the whole site. For a locale-only comparison, use that locale's complete expected set and its matching XML file. Do not compare a docs-only list against a whole-site sitemap and label every blog URL a defect.

For example:

https://docs.example.com/help/
https://docs.example.com/help/docs/install/
https://docs.example.com/help/docs/authentication/

Save this complete Python 3 script as compare_sitemap.py. It reads local XML files, changes nothing, and requires no third-party packages:

import argparse
from collections import Counter
from pathlib import Path
from urllib.parse import urlsplit
import xml.etree.ElementTree as ET

NS = '{http://www.sitemaps.org/schemas/sitemap/0.9}'


def checked_url(value):
    value = value.strip()
    parsed = urlsplit(value)
    if (parsed.scheme not in ('http', 'https') or not parsed.netloc
            or parsed.fragment or any(c.isspace() for c in value)):
        raise ValueError('Invalid absolute URL: ' + repr(value))
    return value


def compare(manifest, files):
    expected = [
        checked_url(line) for line in Path(manifest).read_text(
            encoding='utf-8-sig').splitlines()
        if line.strip() and not line.lstrip().startswith('#')
    ]
    if not expected:
        raise ValueError('Expected URL manifest is empty')
    if len(expected) != len(set(expected)):
        raise ValueError('Expected URL manifest contains duplicates')

    actual = []
    for filename in files:
        root = ET.parse(filename).getroot()
        if root.tag == NS + 'sitemapindex':
            raise ValueError(
                filename + ': supply the referenced child urlset files')
        if root.tag != NS + 'urlset':
            raise ValueError(filename + ': expected a sitemap urlset')
        entries = root.findall(NS + 'url')
        if not entries:
            raise ValueError(filename + ': sitemap contains no URL entries')
        for entry in entries:
            locations = entry.findall(NS + 'loc')
            if len(locations) != 1 or not locations[0].text:
                raise ValueError(filename + ': missing or repeated loc')
            actual.append(checked_url(locations[0].text))

    counts = Counter(actual)
    problems = {
        'missing': sorted(set(expected) - set(actual)),
        'unexpected': sorted(set(actual) - set(expected)),
        'duplicates': sorted(url for url, count in counts.items() if count > 1),
    }
    print('Expected:', len(expected), 'XML entries:', len(actual))
    for label, urls in problems.items():
        print(label + ':', len(urls))
        for url in urls:
            print('  ' + url)
    return 1 if any(problems.values()) else 0


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('manifest')
    parser.add_argument('sitemaps', nargs='+')
    args = parser.parse_args()
    try:
        result = compare(args.manifest, args.sitemaps)
    except (OSError, ValueError, ET.ParseError) as error:
        parser.exit(2, 'Input error: ' + str(error) + '\n')
    raise SystemExit(result)

Run it against the reviewed build output:

python compare_sitemap.py expected-urls.txt build/sitemap.xml

You can pass several downloaded or generated urlset files after the manifest when checking their combined scope. For a sitemap index, obtain its referenced child files first; the script deliberately does not follow network URLs. Avoid passing the same file twice.

Exit code 0 means the exact URL sets match and there are no duplicate XML entries. Code 1 reports a mismatch; code 2 reports invalid input. If authentication/ is missing and preview-login/ is unexpected, the total can still be three. That is why a count-only comparison fails.

The script preserves hostname, path, query string, and trailing-slash differences. It does not crawl pages, validate the full sitemap schema, check lastmod, or establish indexability. An incorrect manifest can also produce a clean comparison, so review the manifest as a release artifact.

Resolve every missing, unexpected, or duplicate entry before accepting the inventory comparison.

3. Check exclusions without bypassing defaults

When a URL is missing, identify which layer removed it before adding it back. A production route, a sitemap entry, and an indexable page are separate things.

The docs plugin front matter reference defines draft and unlisted behavior. Neither hiding a page nor removing its sitemap entry provides access protection. Put genuinely private documentation behind appropriate authentication.

In the current sitemap generation source, default generation filters route paths against ignorePatterns and build-time noindex metadata. A header added by your host after the build is outside that pipeline, so inspect the live response separately.

If you need an additional exclusion, preserve the default generator before filtering its output. For example, replace the sitemap object inside the existing classic preset with this object if the retired guide and its descendants are deliberately excluded:

sitemap: {
  filename: 'sitemap.xml',
  lastmod: null,
  createSitemapItems: async (params) => {
    const {defaultCreateSitemapItems, ...rest} = params;
    const items = await defaultCreateSitemapItems(rest);
    const excluded = '/help/docs/retired-guide';
    return items.filter((item) => {
      const pathname = new URL(item.url).pathname;
      return pathname !== excluded &&
        !pathname.startsWith(excluded + '/');
    });
  },
},

The path boundary avoids accidentally excluding /help/docs/retired-guide-new/. This example assumes the /help/ base path from check1; change it to match your actual emitted routes. It is an excerpt, not a second plugin registration or a complete configuration file.

For simple route globs, ignorePatterns may be enough. Remember that it matches route paths, which can include baseUrl; a pattern copied from a root-hosted site may miss a subpath deployment. In a callback, by contrast, item.url is an absolute URL. Inspect the actual values instead of assuming both mechanisms receive the same string.

Do not remove every old version just because it is old. A supported release with distinct instructions may deserve its own searchable page. Resolve duplicate or superseded pages using a deliberate canonical policy, and make the sitemap agree with that policy.

Keep an intentional reason for each exclusion, and confirm that custom code does not restore pages the default pipeline would omit.

4. Validate lastmod from its source

A sitemap without <lastmod> is not automatically broken. The native plugin's default is lastmod: null, which omits the field. You can choose 'date' or 'datetime' in the existing sitemap options when you have reliable update information.

Enabling the option does not guarantee a date for every route. The plugin uses route metadata such as lastUpdatedAt or a source file path from which version-control information can be obtained. Custom routes or third-party content plugins may not provide that information. The current item-generation implementation shows the order: an explicit timestamp takes precedence; an explicit null stops the file-history fallback.

For file-based documentation, inspect a recently changed page and a page whose substantive content has not changed. Compare their XML dates with the actual source history or a deliberate last_update.date front matter value. If the build relies on Git history, check the checkout used by CI, not just your full local repository:

git rev-parse --is-shallow-repository
git log -1 --format=%cI -- docs/install.md

Use the actual source path for that route. A custom slug or versioned document can make URL-to-filename guesses wrong. If history is shallow or the route metadata is unavailable, correct the date source or omit the field rather than filling every entry with the deployment timestamp.

Google says it uses lastmod when the values are consistently verifiable and accurate. The date should reflect the last significant page update, such as a change to main content, structured data, or links. Changing a copyright year alone is not such an update. Google's sitemap guidance also says it ignores <priority> and <changefreq> values, so tuning those fields is not a substitute for accurate URLs and dates.

Publish dates only when their source is defensible. An understood omission is more useful than invented freshness.

5. Verify the deployed XML and destination pages

Repeat the comparison after deployment. A correct local file cannot detect an upload omission, an outdated cached response, or a rewrite that serves the site's HTML shell at the sitemap URL.

For the hypothetical /help/ deployment, download the public response and retain its headers. On Windows, use curl.exe explicitly; on systems with curl installed as curl, use that command name:

curl.exe --fail-with-body --location --dump-header sitemap-headers.txt --output deployed-sitemap.xml --write-out "%{http_code} %{url_effective}\n" https://docs.example.com/help/sitemap.xml
python compare_sitemap.py expected-urls.txt deployed-sitemap.xml

Replace the example URL and filenames with your real deployment. If you renamed the file or deploy locales separately, fetch each actual sitemap and compare the corresponding inventory. Review the final URL and every redirect hop in the saved headers. A 200 response is insufficient if the body is HTML or an old release's XML.

Confirm the body parses as a sitemap urlset, or as a sitemap index whose referenced children you inspect. Check that the content type is appropriate for XML and that the body corresponds to the reviewed release. The comparison script catches an HTML fallback and URL-set differences; inspect dates and other XML changes separately when they matter. For large sites, Google's per-file limits are 50,000 URLs or 50MB uncompressed; split larger inventories and verify the child files.

Next, inspect representative destinations from each release group: a new guide, an unchanged guide, an older supported version, and each locale or additional docs instance you publish. Fetch their actual page HTML and response headers, then check:

• The URL reaches the intended documentation page with the expected content, rather than a soft error or login page.
• The final URL, canonical tag, sitemap entry, and internal link form agree.
• Neither a robots meta tag nor X-Robots-Tag unexpectedly applies noindex.
• The page is not unintentionally blocked from crawling by the site's robots rules.

Follow up on excluded pages too. A URL deliberately omitted because it canonicalizes elsewhere should express that canonical policy on the actual page. A page meant to be noindexed needs the appropriate directive; removing it from a sitemap alone does not remove it from search results.

Finally, verify discovery using the real sitemap location. If the site's root robots.txt advertises the file, confirm its Sitemap: URL is accurate. If you use Search Console, submit or inspect the actual sitemap in the property that covers its public URLs. A successful submission confirms receipt, not indexing of every listed page. Our XML sitemap guide covers that broader discovery workflow.

Keep this short release record so the next maintainer can reproduce the decision:

Treat the sitemap as verified when the deployed file and sampled destinations support the same release decision as the build artifacts. If they disagree, investigate the configuration, build, or delivery layer responsible.