What does Google say about SEO?

Is Google refusing to index your 'Discovered' pages? Here's why.

Martin Splitt Aug 20, 2024 ★★★

The most common reason why pages remain in 'Discovered - currently not indexed' status is content quality. When Google detects a pattern of low-quality or poor content on pages, those pages can be removed from the index or remain in 'Discovered' stat...

Domain Age & History Content Crawl & Indexing AI & SEO

Is Google really ignoring entire sections of your site because of a low-quality pattern?

Martin Splitt Aug 20, 2024 ★★★

If Google Search detects a pattern of URLs with low-quality content on your site, it can decide to skip these URLs entirely, leaving them in 'Discovered' status. Googlebot knows these pages exist but chooses not to process them.

Domain Age & History Content Crawl & Indexing AI & SEO Domain Name

Does internal linking alone really guarantee your undiscovered pages will get indexed?

Martin Splitt Aug 20, 2024 ★★★

If you want pages with the status 'Discovered - currently not indexed' to be indexed, you should rework the content to improve its quality and ensure your internal linking connects this content to other parts of your existing content.

Domain Age & History Content Crawl & Indexing AI & SEO Pagination & Structure

Does Googlebot Really Collect Your Links Instead of Following Them?

Gary Illyes Aug 13, 2024 ★★★

Contrary to what Google states in its official documentation, Googlebot doesn't follow links in real-time but collects them for later processing. This clarification was shared by Gary Illyes in the Search Off The Record podcast: "On our site, we keep...

Crawl & Indexing AI & SEO Links & Backlinks PDF & Files

Should You Ditch JavaScript for SEO Because of Google's Rendering Costs?

Martin Splitt Aug 13, 2024 ★★

On LinkedIn, Martin Splitt explained that crawling and rendering, particularly JavaScript pages, represented a potentially significant cost for Google, while specifying that there was no need to worry about rendering being expensive.

Domain Age & History Crawl & Indexing AI & SEO JavaScript & Technical SEO Links & Backlinks Social Media

Are URL Parameters Really Threatening Your Site's Google Crawlability?

Gary Illyes Aug 13, 2024 ★★★

Gary Illyes warned that URL parameters could cause crawling issues for Google, particularly by generating an infinite number of URLs for a single page: "technically, you can add an almost infinite number of parameters to any URL, and the server will ...

Domain Age & History Content Crawl & Indexing E-commerce AI & SEO Links & Backlinks Domain Name

Does intensive crawling really guarantee a high-quality website?

Gary Illyes Aug 08, 2024 ★★★

Google may crawl a site more extensively for various reasons (quality content, hacks, new URLs, calendar scripts). High crawl volume does not automatically mean the site is high quality. Conversely, reduced crawling may indicate low-quality content o...

Domain Age & History Content Crawl & Indexing AI & SEO Domain Name

Does forcing Google to crawl more pages actually boost your search rankings?

Gary Illyes Aug 08, 2024 ★★★

Forcing Google to crawl more of your website (for example via robots.txt) won't make your site rank better in search results. Content quality must come first for Google to naturally increase crawl frequency.

Domain Age & History Content Crawl & Indexing AI & SEO

Can you really boost your site's crawl budget by contacting Google?

Gary Illyes Aug 08, 2024 ★★★

Requests submitted via the Googlebot issue report form cannot be used to increase a site's crawl volume. This form is intended solely to report over-crawling problems or server overload issues.

Crawl & Indexing AI & SEO Search Console

Why does Google crawl some sites more frequently than others?

Gary Illyes Aug 08, 2024 ★★★

Crawl volume is determined by the server's technical capacity to handle requests and by the quality/usefulness of content for users. These two aspects define the frequency and intensity of crawling.

Content Crawl & Indexing

Do URL parameters really create an infinite crawl space for Google?

Gary Illyes Aug 08, 2024 ★★

URL parameters can generate a nearly infinite number of versions of the same page. Google must crawl a large sample to determine whether the parameters actually modify the content. Webmasters can use robots.txt to block URL spaces with unnecessary pa...

Domain Age & History Content Crawl & Indexing JavaScript & Technical SEO Domain Name

Why do hashtags and URL anchors complicate Google's crawling process?

Gary Illyes Aug 08, 2024 ★★

URL fragments (hashtags/#) exist only on the client side and Googlebot cannot access them without rendering. This complicates crawling and the discovery of content based on anchors.

Content Crawl & Indexing JavaScript & Technical SEO Links & Backlinks Domain Name

Is Google really hiding the secret to faster indexing in your Crawl Stats?

John Mueller Aug 08, 2024 ★★★

Webmasters should check crawl statistics (Crawl Stats) in Search Console to diagnose crawl problems. Average response time is displayed there: several seconds of response time constitute an objective issue that slows down crawling.

Crawl & Indexing AI & SEO Search Console

Is a slow server response time killing your crawl budget?

John Mueller Aug 08, 2024 ★★★

If a server takes 3 seconds instead of 100 milliseconds to respond, that represents 20 to 30 times more time. Across millions of URLs, this significantly reduces the number of pages Google can crawl. Webmasters must optimize server response time.

Domain Age & History Crawl & Indexing Domain Name

Does Googlebot really crawl links sequentially like a user navigating from page to page?

Gary Illyes Aug 08, 2024 ★★

Contrary to popular belief, Googlebot does not follow links page by page like a user would. It first collects the links and then returns to them independently. This technical distinction is important for understanding crawler behavior.

Domain Age & History Content Crawl & Indexing AI & SEO Links & Backlinks

Should you really optimize crawl budget when Google has unlimited resources?

Gary Illyes Aug 08, 2024 ★★

Google has sufficient resources for crawling. Crawl optimizations (reduction of unnecessary URLs, improvement of response times) primarily benefit websites by allowing Google to crawl genuinely useful URLs rather than low-value content.

Content Crawl & Indexing Domain Name

Is Google Really Pushing the If-Modified-Since Header to Cut Crawl Costs?

Gary Illyes Aug 08, 2024 ★★★

Servers should properly implement the 304 Not Modified response to the If-Modified-Since header. This saves bandwidth and server resources by avoiding retransmitting unchanged content. Currently, this feature is not being used widely enough by websit...

Content AI & SEO

Are sitemaps really essential for optimizing your site's crawl performance?

John Mueller Aug 08, 2024 ★★

Sitemaps have been an important crawl optimization since Google's early days. While some use them incorrectly, they remain a recommended tool to help search engines discover content efficiently.

Content Crawl & Indexing AI & SEO Search Console

Does product variant markup really change the game for e-commerce SEO?

John Mueller Aug 07, 2024 ★★★

Google now offers specific markup to tag product variants (like different colors of the same item), enabling better representation of product variations in search results.

Domain Age & History Discover & News E-commerce AI & SEO

How can you optimize Sitemaps for large e-commerce sites according to Google's latest guidance?

John Mueller Aug 07, 2024 ★★

Google offers specific recommendations on using Sitemaps to optimize large e-commerce sites, with a dedicated article addressing this challenge.

Crawl & Indexing Discover & News E-commerce Search Console