Sitemap.xml Audit Script: Check lastmod, Limits and Duplicates (Next.js Example)

A sitemap is the one file you hand search engines and then forget. That is how you end up with a lastmod that says every page changed at the moment of your last deploy. This post covers what Google's documentation says a sitemap must be, gives you a script that checks it, and shows what the script found on this site's real Next.js build.

Rules come from Google's "build a sitemap" page, read on 2026-10-16. I did not find lastmod guidance on Google's sitemaps overview page; the lastmod and ignored-tag statements below come from the build page.

What does Google require from a sitemap?

• Size: each file up to 50MB uncompressed or 50,000 URLs. Split beyond that. A sitemap index is optional.
• Formats: XML, RSS 2.0 or Atom 1.0 feeds, or a plain text file with one URL per line.
• Absolute URLs: relative paths such as /mypage.html are not acceptable.
• UTF-8 encoding, with XML tag values entity escaped.
• Canonical URLs only: list the URL you want shown in results, not every variant.
• Location: Google recommends the site root.
• Ignored tags: priority and changefreq are ignored.
• lastmod: used only if values are consistently and verifiably accurate.
• A hint, not a guarantee that Google will crawl.

The audit script

It parses the XML and reports the things above that a script can check. The final two checks are my own heuristics for spotting dishonest lastmod values: it prints the most common timestamp to the second and the number of distinct dates.

raw = Path(path).read_bytes()
raw.decode("utf-8")  # raises if not UTF-8
root = ET.fromstring(raw)
urls = root.findall(f"{NS}url")
...
print(f"urls: {len(locs)} (limit 50,000) | size: {len(raw)/1e6:.2f} MB (limit 50 MB)")
print("relative or non-http URLs:", sum(1 for l in locs if urlparse(l).scheme not in ("http", "https")))
print("duplicate URLs:", len(locs) - len(set(locs)))
c = Counter(lm[:19] for lm in lastmods)
top, n = c.most_common(1)[0]

What did it find on this site?

I built the site (npm run build) and ran the script on the generated sitemap body on 2026-10-16:

urls: 756 (limit 50,000) | size: 0.21 MB (limit 50 MB)
relative or non-http URLs: 0
duplicate URLs: 0
hosts: {'masterprompting.net': 756}
urls with lastmod: 756 | with priority: 756 | with changefreq: 756 (Google ignores both)
most common lastmod (to the second): 2026-10-09T00:00:00 on 214 URLs
distinct lastmod dates: 29

The structural checks pass: well under the limits, no duplicates, one host, no relative URLs. Two things are worth a closer look.

1. Every URL carries priority and changefreq. Google ignores them, so they do nothing for Google. They are harmless but they are noise, and they set false expectations when someone edits the numbers hoping to change crawling.

2. 132 URLs have a lastmod equal to the build moment. I counted timestamps that were not midnight. All of them were within a few milliseconds of 2026-10-16T05:08:26:

The sitemap source builds static pages with lastModified: new Date(), so these stamps change on every build, whether or not the page changed. The other 624 URLs use real content dates, taken from each post's updated or date field, and they look healthy: the whole file has 29 distinct lastmod dates, one of which is the build date.

Is that a problem? Google says it uses lastmod only if values are consistently and verifiably accurate. About 17% of this sitemap (132 of 756 URLs) changes on every deploy for unchanged pages, which is the opposite of verifiable. I cannot say how Google weighs this for this site, and I found no primary source on partial penalties, so I will not claim harm. The cautious reading is that Google may learn to distrust the field.

I did not change the site's code in this post. The simple fixes are either to use the real last-edited date for those pages or to leave lastModified out for pages with no meaningful date. Omitting it is better than lying.

How do you fix lastmod in Next.js?

In app/sitemap.ts, each entry is an object. For content pages, derive lastModified from the file's real date (frontmatter updated, or the Git commit date). For listing pages like /blog, use the newest date among the items they list. For truly static pages, omit the field.

// Sketch: listing page takes the newest item date instead of "now"
const newest = blogPosts.map((p) => lastModified(p)).sort().at(-1);
{ url: `${BASE_URL}/blog`, lastModified: newest ? new Date(newest) : undefined }

That sketch is untested on this codebase; the script above is how you would check the result after rebuilding.

What else to check before you submit?

1. Every URL returns 200 and is the canonical version. A sitemap full of redirects or noindex pages wastes the hint.
2. Pages you block with noindex are not listed. The snippet controls post explains that tag.
3. The sitemap is referenced in robots.txt and submitted in Search Console, where you can track each file.
4. For fast notifications on Bing and others, add IndexNow. The IndexNow FAQ recommends using sitemaps alongside it.

For the big picture of what Google says about AI features and crawling, read the AEO checklist, and to track what happens after a change, use the low-CTR script.

What I could not verify

• How Google treats a sitemap where part of the lastmod values are unreliable.
• Whether the URLs in the sitemap all return 200. The script checks structure, not live status.
• Image sitemap extension details. This site adds image entries, which I did not audit.