Featured image: “Rows of Bookshelves” by Parihav, licensed under CC BY-SA 3.0.
“Discovered — currently not indexed” and “Crawled — currently not indexed” sound like the same problem with two labels, but Google treats them very differently. The first means Google found the URL and chose not to spend crawl resources on it yet, typically because it expects the crawl to strain the site; the second means Google already read the page and decided, after reading it, not to index it. A site publishing a high volume of AI-assisted posts is likely to see both, but the fix for one has almost nothing to do with the fix for the other.
Two Statuses, Two Different Decisions
Google’s own Page Indexing report documentation defines “Discovered — currently not indexed” as a URL “found by Google, but not crawled yet,” and specifically notes that the crawl is rescheduled because Google expects crawling it right now would overload the site — which is why the report’s last-crawl-date field stays empty for these URLs. “Crawled — currently not indexed,” by contrast, means Google already visited the page and decided not to index it, with the note that it “may or may not be indexed later.” One status is about timing and server load; the other is about what Google found when it actually read the content.
“Discovered”: A Capacity Decision, Not a Verdict on Quality
Whether a known URL gets crawled soon depends on what Google calls crawl budget, which Google defines as “the set of URLs that Google can and wants to crawl,” shaped by two separate factors: a crawl capacity limit (how much server load Google is willing to put on your site) and crawl demand (how much Google wants to recrawl based on perceived inventory, popularity, and staleness). A burst of new posts published faster than the site’s crawl capacity limit can absorb will sit in “Discovered” simply because Google is pacing itself, independent of whether those posts are any good.
Google’s crawl budget guidance also names concrete things that eat into that capacity for everyone on the site, which is worth checking before assuming a publishing-volume problem: duplicate content competing for the same crawl slots, URLs for removed or re-sorted pages that should be blocked in robots.txt instead of crawled repeatedly, soft 404s that keep getting recrawled, and long redirect chains. None of these are about whether any single new post is well-written; they are about whether the crawler is wasting its visits on pages that were never going to be worth indexing in the first place.
“Crawled”: This One Is Actually About Quality
Once a page has a real crawl date and still lands in “Crawled — currently not indexed,” crawl capacity is no longer the explanation, because Google already spent the resources to read it. Google’s guidance here is intentionally vague about specific causes, saying only that the page may be indexed later, but the status sits downstream of the same content-quality evaluation covered in Google’s Helpful Content guidance and Scaled Content Abuse policy: pages assessed as adding little beyond what already exists for a query are the ones most likely to be crawled, evaluated, and then left out of the index rather than queued for a later recrawl.
For the specific policy that most directly drives this outcome at high publishing volume, see our checklist for auditing a site against Google’s Scaled Content Abuse policy, and for what the fix looks like on an actual page, rewriting a thin AI-generated page into one that passes the Helpful Content guidelines walks through a real before-and-after.
Why High-Volume AI-Assisted Publishing Tends to Produce Both at Once
Publishing many posts in a short window stresses both halves of this problem simultaneously, and that is precisely what makes the two statuses easy to confuse. The sheer volume can push some URLs into “Discovered” purely on crawl-capacity grounds — a pacing issue that resolves on its own as Google works through the queue. At the same time, if a meaningful share of those posts share a near-identical structure, intro pattern, and depth, the ones that do get crawled quickly are the ones most likely to land in “Crawled — currently not indexed,” because they read as duplicative of each other even when no two are copied text. Seeing a large batch of “not indexed” URLs after a publishing surge is not, by itself, evidence that the content is low-quality; checking the crawl date on those specific URLs is what tells you which problem you actually have.
A Short Diagnostic Checklist
- Check the last crawl date on the affected URLs in the Page Indexing report. Empty means “Discovered” (a capacity/timing issue); a real date means “Crawled” (a quality evaluation already happened).
- For “Discovered” URLs, check server response times and error rates during the publishing window. Slow responses or 5xx/429 errors lower the crawl capacity limit itself, independent of content.
- For “Discovered” URLs, confirm the sitemap is current and submitted. A stale sitemap after publishing through the REST API is a separate, common cause — see our diagnosis of a stale Rank Math sitemap after REST API publishing if posts were published that way.
- For “Crawled” URLs, compare the page against the top-ranking results for its target query. If it restates what those pages already say without a distinct angle, structure, or original element, that is the Scaled Content Abuse and Helpful Content signal, not a crawl-budget issue.
- Don’t resubmit “Crawled” URLs repeatedly through URL Inspection. Google’s own guidance for this status says resubmission isn’t necessary; repeated requests don’t change the evaluation that already happened.
What Actually Moves a Page Out of Each Status
A “Discovered” URL typically clears on its own once crawl capacity catches up with the backlog, and the only lever a site owner has is not adding to the backlog — keeping server response times fast, fixing soft 404s, and blocking pages that never needed to be crawled in the first place, like filtered or duplicate URL variants. A “Crawled” URL, in contrast, generally needs the page itself to change: a substantive rewrite that adds something the top-ranking pages for that query don’t already cover, not a resubmission or a sitemap ping. Treating both with the same fix — usually “wait and resubmit” — is why this particular pair of statuses causes more wasted effort than almost any other pairing in the Page Indexing report. For sites that also serve AI crawlers and answer engines rather than only traditional search, it’s worth checking that a correctly configured llms.txt file isn’t being treated as a substitute for either fix, since it affects neither crawl budget nor the Helpful Content evaluation.
The Role of Internal Linking in Crawl Demand
Crawl demand is not just a function of how popular a site is overall; it also depends on how easy Google’s crawler finds it to reach a given page at all. A new post with no internal links pointing to it, sitting several clicks deep from the homepage, signals lower priority than one linked from several already-indexed, already-crawled pages on related topics. This is a case where good internal linking does double duty: it helps a human reader find related material, and it gives the crawler a second and third path to a new URL beyond the sitemap alone, which matters specifically for pages stuck in “Discovered” rather than “Crawled,” since that status is exactly the one crawl-priority signals can influence.
What This Looks Like on a High-Volume Publishing Schedule
Picture a site that moves from publishing a handful of posts a week to ten a day. In the days right after that change, checking the Page Indexing report will typically show a spike in “Discovered — currently not indexed,” most of it with an empty crawl date, because Google’s crawl capacity limit for that host hasn’t caught up with the new volume yet; this is the pacing effect described above, and it tends to thin out over one to a few weeks if server response times stay fast and nothing else gets in the way. A smaller, and more worrying, slice of that same spike will show an actual crawl date and sit in “Crawled — currently not indexed” instead — and if those happen to be the posts published in the biggest daily batches with the most repetitive structure, that correlation is itself a diagnostic clue, not a coincidence to explain away. The fix for the first group is patience plus the server and sitemap hygiene covered above; the fix for the second is to slow down enough that each post can clear the bar the first group never had to meet.
One more detail from Google’s own crawl budget guidance is easy to miss: keeping sitemaps current and including an accurate <lastmod> value on every entry helps Google prioritize what to recrawl and when, which feeds directly into the crawl-demand side of the crawl budget calculation described above. A sitemap that lists every post with the same unchanged lastmod, or one that silently falls behind after a bulk publish, removes a signal Google explicitly says it uses — which is one more reason a stale sitemap and a publishing surge compound each other rather than being two unrelated problems.
