Google’s John Mueller answered a question in a Google Office Hours Hangout about a Search Console but where URLs were listed as excluded but when the URL is examined the web page was listed as indexed.
Google’s John Mueller said that he’s seen reports of this anomaly and that he had an idea of what it might be.
A person asked a question about an issue where Google reports that pages are not indexed but when examined another report says they are indexed.
This issue is making it difficult for the person to accurately track crawling and indexing statistics for the site.
The person asking the question explained the problem:
“We have like a very large number of Crawled Not Indexed Pages listed under Excluded.
But then when we click into them most of these seem to have been converted into indexed pages.
So we’re really unable to accurately track how improvements to our site are impacting which pages are being indexed.
And I was curious I guess about the timeline in that.
We’re concerned it’s impacting our crawling budget.”
Continue Reading Below
The person asking the question was concerned that the crawled but not index error was causing an issue with their crawl budget.
A crawl budget is the amount of URLs that Google allots to crawl on a site.
The crawl budget is calculated partly on the servers ability to serve pages. This is called the Crawl Capacity Limit.
If a server has a hard time serving pages…