Does Google really crawl some URLs only once every six months?

Does Google crawl all pages at the same frequency?

No. Google adjusts its crawl frequency based on what it assumes to be a URL's stability. If a page is perceived as rarely modified, it will be visited less frequently — sometimes only once every six months.

This classification relies on predictive signals: modification history, type of content, position in the site hierarchy, freshness signals. Google does not guess randomly — it infers from its previous visits and behavioral analysis of the site.

What triggers more frequent recrawling?

URLs that change regularly, generate traffic, or belong to strategic sections (blog, news, updated product pages) are crawled more often. Google also monitors freshness signals: lastmod tag in the XML sitemap, date mentions in the content, update patterns.

Conversely, a technical page buried in the hierarchy, with no traffic or changes for months, will be relegated to the bottom of the pile. Google conserves its crawl budget by prioritizing what moves and what matters.

Why does this logic pose problems for some sites?

Because it creates a vicious cycle: a poorly crawled page does not get updated in the index, so it does not rise in search results, thus it generates no traffic, leading Google to crawl it even less.

This is particularly problematic for sites that update existing content without creating new URLs. If Google does not come back, the SEO improvements are never indexed. The result: wasted effort.

• Google classifies URLs based on their presumed modification frequency, not their actual importance
• Some pages may be crawled only once every six months if they are deemed stable
• Freshness signals (sitemap, lastmod, dated content) directly influence crawl frequency
• A poorly crawled page enters a vicious cycle: no indexing = no traffic = less crawling
• The crawl budget is a limited resource: Google optimizes its visits based on its predictions

Is this statement consistent with field observations?

Yes, and it's even an understatement. On sites with deep hierarchies or low-dynamic product catalogs, we regularly observe aberrant indexing delays — sometimes several months for a simple price or description update.

John Mueller is stating the obvious here, but he deserves credit for saying it clearly: Google does not crawl everything, all the time. What is missing from this statement is a precise definition of the classification criteria. What causes a URL to switch from weekly to semi-annual crawling? [To be verified] due to the lack of publicly available actionable data.

What nuances should be added to this rule?

First, this logic mainly applies to sites with a tight crawl budget — which means most sites. An authoritative site with good internal linking and regular backlinks will have less of this issue: Google is more generous with resources.

Then, there are ways to push the issue: submitting the URL via Search Console, including a recent lastmod in the XML sitemap, generating organic or referral traffic to the page. But these maneuvers are not officially documented, so they rely on field experience and empirical observation.

In which cases does this rule not apply?

For news sites or fresh content platforms (media, active blogs), Google automatically increases its crawl frequency. The signal of editorial freshness is detected and rewarded.

Similarly, sites with a high rate of technical updates (e-commerce with stock rotation, classified ad aggregators) benefit from more frequent crawling — provided that the XML sitemap is well configured and freshness signals are sent correctly. Otherwise, even a dynamic site can end up being under-crawled.

What should be done concretely to speed up crawling?

First action: submit modified URLs via the URL Inspection tool in Search Console. It is the most direct way to force an immediate recrawl. Don't just wait for Google to come by itself.

Next, optimize your XML sitemap by including only strategic URLs, with up-to-date lastmod tags. A sitemap polluted by thousands of outdated URLs dilutes the signal and slows down crawling. The fewer URLs there are, the more responsive Google is to those that remain.

What mistakes should be avoided to prevent worsening the problem?

Do not multiply unnecessary URLs: sorting parameters, filters, session variations. Every URL consumes crawl budget. If Google spends its time on duplicates or pages without value, it crawls your strategic content less often.

Also avoid leaving orphan pages without internal linking. A URL that is invisible in the hierarchy will only be visited if Google finds it elsewhere (sitemap, external backlink). This can take months. Always link your updated content from active pages.

How can I check if my site is being crawled correctly?

Check your server logs: identify the URLs crawled by Googlebot, their frequency, response codes. A tool like Screaming Frog Log Analyzer or OnCrawl allows you to cross-reference crawl data with your site hierarchy and sitemap.

In Search Console, monitor the coverage report and exploration statistics. If you see URLs “discovered but not crawled” that stagnate for weeks, it's a sign that Google has deprioritized them. Act accordingly.

• Manually submit modified URLs via the Search Console inspection tool
• Optimize the XML sitemap: only include strategic URLs, with up-to-date lastmod tags
• Strengthen internal linking to updated content to increase their visibility in the hierarchy
• Analyze server logs to identify under-crawled URLs and adjust your strategy
• Avoid multiplying unnecessary URLs (parameters, filters, duplicates) that dilute crawl budget
• Monitor the coverage report in Search Console to detect “discovered but not crawled” URLs