Gary Illyes

★★ Does Google really index all your XML files?

Google selectively indexes XML files. Sitemaps and podcast feeds can be indexed, but RSS and Atom feeds generally cannot. The decision depends on the declared XML namespace and the content-type header...

Sep 08, 2022 ⚡ Analysis available

★★ Can you really get JSON and plain text files indexed in Google search results without metadata?

JSON and text files can be indexed and served in search results if Google has enough context. The lack of internal titles and metadata makes these files difficult to rank, but external links with desc...

Sep 08, 2022 ⚡ Analysis available

★★★ Do you really need to publish every day to improve your Google SEO?

Publishing daily or at a specific frequency doesn't help you rank better in Google search results. However, the more pages you have in Google's index, the more your content can appear in search result...

Sep 07, 2022 ⚡ Analysis available

★★★ Do images really consume your crawl budget at the expense of your strategic pages?

Googlebot and its variants (Images, etc.) share a single crawl budget. If you have many images, Googlebot Images can use a portion of the budget that could have been used by Googlebot. This is not a c...

Sep 07, 2022 ⚡ Analysis available

★★ Does audio content on a page actually influence your Google ranking?

Audio content, in itself, does not play a role in ranking text-based results. There can be indirect effects if users find the page more useful and recommend it more....

Sep 07, 2022 ⚡ Analysis available

★★★ Are Google's algorithm updates really that different from penalties?

Named updates published on the Search Central Rankings Updates page are not penalties. They are adjustments to ranking algorithms to deliver higher quality and more relevant results. If a site's ranki...

Sep 07, 2022 ⚡ Analysis available

★★★ How Does Google Crawl Your Site by Sections and Why Are Some Areas Explored More Often Than Others?

Gary Illyes explained during a recent "Search Off the Record" podcast that Google can analyze certain parts or sections of sites and thus crawl certain areas more frequently while visiting others less...

Sep 05, 2022 ⚡ Analysis available

★★★ Should you really be concerned about crawl budget for your website?

The vast majority of websites (over 90%) don't need to worry about crawl budget. It's a rare problem that only affects very large sites or sites with specific needs....

Aug 25, 2022 ⚡ Analysis available

★★★ What's Google's real definition of crawl budget—and which levers can actually move the needle?

Google defines crawl budget as the number of URLs that Googlebot can and is willing to crawl for a given site. This definition rests on two factors: crawl capacity (not overloading the server) and cra...

Aug 25, 2022 ⚡ Analysis available

★★ Is crawl budget a concept invented by Google or by SEO professionals?

For a long time, Google claimed it didn't have a concept of crawl budget. Following discussions within the SEO community, Google created a definition by working with several internal teams to map exis...

Aug 25, 2022 ⚡ Analysis available

★★★ Does Google really index only a fraction of the web because of storage costs?

Google does not have infinite storage capacity. Indexing requires storage (hard drives, memory, SSDs) that costs money. Google therefore does not index all available content on the Internet, only what...

Aug 25, 2022 ⚡ Analysis available

★★★ Can you really manage your crawl budget from Google Search Console?

Webmasters can indirectly control their crawl budget through the crawl statistics in Search Console. They can limit the maximum number of requests per second (QPS) that Googlebot makes to their site t...

Aug 25, 2022 ⚡ Analysis available

★★ Does HTTP/2 really boost your crawl budget?

Enabling HTTP/2 on your server significantly improves crawl budget utilization. HTTP/2 allows Googlebot to open a single connection and stream requests instead of opening multiple simultaneous connect...

Aug 25, 2022 ⚡ Analysis available

★★★ What does your "discovered but not crawled" URL status really reveal about your site?

If a large proportion of URLs appears as "discovered but not crawled" in Search Console, this indicates either a content quality issue (Google doesn't think users are searching for this content), or a...

Aug 25, 2022 ⚡ Analysis available

★★ Are you wasting your crawl budget on JavaScript files that add no value?

If a significant proportion of crawl budget (35% or 90%) is consumed by JavaScript files that add no content, it is recommended to consolidate these files or use X-Robots-Tag headers to prevent their ...

Aug 25, 2022 ⚡ Analysis available

★★★ Are 404s and robots.txt Really Wasting Your Crawl Budget?

HTTP status codes 404 and 410, as well as URLs blocked by robots.txt, do not consume crawl budget because Google only receives the status code without content. Conversely, soft 404s (pages that return...

Aug 25, 2022 ⚡ Analysis available

★★ Should you block your decorative JavaScript files to optimize your crawl budget?

If JavaScript files are purely decorative and add neither content nor value to the page rendering, they can be blocked via robots.txt or X-Robots-Tag. Rendering will fail for these resources but this ...

Aug 25, 2022 ⚡ Analysis available

★★★ Why is Google now rejecting certain directives in your robots.txt file?

The robots.txt file should only be used to control crawling. Google removed certain directives like noindex from the robots.txt parser because they don't concern crawling. Indexing and serving must be...

Aug 04, 2022 ⚡ Analysis available

★★★ Can Google really index your URLs blocked by robots.txt without you being able to do anything about it?

If a page blocked by robots.txt disallow is very popular on the internet, Google can index the URL without the content. The result will appear without a snippet but with the title. In this case, a met...

Aug 04, 2022 ⚡ Analysis available

★★★ Does Google really process HTTP status codes during the crawl phase, not after?

404, 410, and 403 codes are detected during the crawling phase. Google sends a signal to the indexing system indicating that the URL no longer exists, and indexing then decides to remove it. The crawl...

Aug 04, 2022 ⚡ Analysis available