Google Can’t Index What It Can’t Find: A WordPress XML Sitemap Guide

Search engines don’t magically know every page on your site exists. They find pages by following links and by reading a map you hand them — an XML sitemap. When that map is missing, wrong, or pointing somewhere it shouldn’t, whole sections of your site can sit unseen for months while you wonder why they never rank.

An XML sitemap is a plain file that lists the URLs you want search engines to know about, along with a little metadata about each one. It won’t force anything into the index, and it isn’t a ranking trick. What it does is make discovery reliable instead of leaving it to chance — especially for a big site, a new site with few backlinks, or pages buried several clicks deep.

The last SEO audit we ran turned up a site with about 400 published pages and only 60 in Google’s index. The sitemap existed, it validated, it returned a clean 200. It had also been pointing at the client’s old staging domain for the better part of a year, so every URL in it 404’d the moment a crawler tried to follow it. Nobody had looked at the actual contents since launch. That’s the kind of quiet failure this guide is meant to help you catch.

What an XML Sitemap Actually Does

Think of it as a directory you give to search engines. Each entry is a URL you consider worth crawling, and the file as a whole says “here is the canonical set of pages on this site.” Google, Bing, and the AI crawlers can read it in one request instead of hoping to stumble onto everything through internal links.

It matters most in a few specific situations. A brand-new site has almost no inbound links, so a sitemap is often the fastest way for a crawler to learn the site exists at all. A large site — thousands of products, posts, or listings — has pages that live many clicks from the homepage, where crawlers may never reach on their own. And a site with a thin internal-linking structure leaves orphan pages that nothing points to; the sitemap is sometimes the only path to them.

What a sitemap does not do is guarantee indexing. Google is explicit about this: a sitemap helps it discover URLs, but it decides on its own which pages to crawl and index. Treat the file as a strong hint, not a command.

WordPress Already Builds One For You

Here’s the part a lot of site owners miss: since WordPress 5.5, released in August 2020, core generates an XML sitemap automatically. You don’t need a plugin to have one. Visit yoursite.com/wp-sitemap.xml and you’ll almost certainly see it.

That URL is a sitemap index — a table of contents that links out to smaller sitemap files grouped by type. By default WordPress splits things into chunks of up to 2,000 URLs per file (controlled by the wp_sitemaps_max_urls filter), which keeps every individual file well under the protocol’s ceiling. It also adds a Sitemap: line to your virtual robots.txt pointing crawlers straight to the index, so search engines can find it without you lifting a finger.

What the default sitemap includes

Out of the box, the core sitemap covers your homepage plus your published posts, pages, and any public custom post types, along with taxonomy archives (categories, tags, and public custom taxonomies) and author archives. Any post type marked non-public is left out automatically. Per-page noindex exclusion, though, is handled by SEO plugins like Yoast or Rank Math rather than by core itself — the only site-wide switch core gives you is the “Discourage search engines” setting, which turns the whole sitemap off.

That default is genuinely good enough for a lot of small sites. If your only goal is “make sure Google can find my posts and pages,” the built-in file does that job the day you install WordPress.

When You Need More Than the Default

The core sitemap is deliberately minimal. It lists URLs and not much else, and it gives you no dashboard, no per-page toggles, and no easy way to exclude a specific page without code. Once your site gets past a couple dozen pages, that’s usually where an SEO plugin earns its place.

Yoast SEO, Rank Math, SEOPress, and AIOSEO all replace the core sitemap with their own — typically at a URL like sitemap_index.xml — and turn off wp-sitemap.xml so you don’t run two competing maps. Their versions add controls the default lacks: excluding individual posts, dropping entire post types or taxonomies, splitting out images or news, and a UI to see what’s included. If you already run one of these plugins for titles and meta descriptions, you already have a more capable sitemap; just confirm which URL is the live one before you submit it anywhere.

One sitemap, not two

The mistake to avoid is running the core sitemap and a plugin sitemap at the same time. Reputable SEO plugins disable the core one for you, but if you’ve hand-rolled something — a snippet in functions.php, an old caching rule — you can end up serving two maps that disagree. Pick one source of truth and make sure everything else points at it.

Submitting Your Sitemap to Google

WordPress advertises your sitemap in robots.txt, and search engines will find it on their own eventually. Submitting it in Google Search Console just makes that faster and gives you a feedback loop.

Add and verify your site in Google Search Console, open the Sitemaps report, and enter the path to your sitemap index — wp-sitemap.xml for core, or sitemap_index.xml if a plugin owns it. Google will report how many URLs it discovered and flag any it couldn’t read. Bing Webmaster Tools has an equivalent panel, and it can import your Search Console setup directly.

After submitting, watch the Pages (Page indexing) report over the following weeks. It tells you which submitted URLs got indexed and, more usefully, why the rest didn’t — “Discovered, currently not indexed,” “Crawled, currently not indexed,” “Excluded by noindex tag,” and so on. That report is where a sitemap problem stops being invisible.

Sitemap Mistakes That Quietly Cost You

Most sitemap failures aren’t dramatic. The file loads, it validates, and it’s still working against you. These are the ones we see most often.

Stale or wrong-domain URLs

The staging-domain problem from the intro is more common than you’d think, and it also shows up after an HTTP-to-HTTPS move or a domain change. If your sitemap lists http:// URLs, or an old domain, or a staging host, every entry is a dead end. Reload the sitemap in a browser after any migration and actually click a few links.

Listing pages you’ve told Google to ignore

A sitemap says “please crawl this,” while a noindex tag says “don’t index this.” Put the same URL in both and you’re sending mixed signals. Core WordPress sidesteps this for the pages it lists because it only includes public content, but the moment a plugin or hand-edited map starts adding URLs, a noindex page can slip into the sitemap. Keep the sitemap and your index directives telling the same story.

Blocking the sitemap in robots.txt

It sounds absurd, but a stray Disallow rule can block the very path your sitemap lives on, or block resources the listed pages depend on. If Search Console says it can’t fetch your sitemap, your robots.txt is the first place to look.

Treating the sitemap as your whole SEO strategy

A sitemap helps a crawler find pages. It does nothing for whether those pages deserve to rank. Thin content, missing meta descriptions, no structured data, and a weak internal-linking structure all still matter. The sitemap gets Google to the door — what’s behind it is on you.

Where Sitemaps Fit in the Bigger Picture

Getting discovered is step one. Once a crawler reaches a page, it still has to understand what the page is and decide it’s worth surfacing. That’s where the rest of your technical SEO comes in: clean titles and descriptions so the result is clickable, and schema.org structured data so search engines and AI models can read the page with confidence instead of guessing.

That’s the layer we spend most of our time on. Our own open-source suite, amplifi.plugins, uses AI to bulk-generate SEO meta and schema.org JSON-LD across an entire WordPress site — the understanding layer that sits right behind the discovery layer a sitemap provides. A sitemap gets your pages found; good meta and schema help them get chosen.

Frequently Asked Questions

Getting found is the first half of SEO. If your pages are discoverable but still not chosen, give every page proper meta and schema in minutes.

Built by amplifi.studio — see also the robots.txt mistake making you invisible to ChatGPT and Claude.