Gary Illyes

★★★ Does Google extract meta robots and canonical tags during indexing rather than at crawl time—and why does this distinction matter for your site?

Meta tags such as meta robots noindex and rel canonical are extracted during the indexing process, when Google parses the content and performs rendering. If Google detects meta robots noindex, the URL...

Aug 04, 2022 ⚡ Analysis available

★★ Can a single noindex tag on an hreflang page contaminate your entire international cluster?

In hreflang clusters, pages can influence each other mutually. A noindex on one page in a cluster can potentially affect the entire cluster in case of detected duplication. It is recommended to add no...

Aug 04, 2022 ⚡ Analysis available

★★ Should you really rely on JavaScript to handle noindex directives?

Adding meta noindex via JavaScript should work in most cases after rendering, but there can be time windows or situations where rendering doesn't happen and the directive wouldn't be seen. It's better...

Aug 04, 2022 ⚡ Analysis available

★★★ Should you use X-Robots-Tag to keep PDFs and binary files out of Google's index?

For binary files like PDFs where it's impossible to add a meta tag, Google supports the X-Robots-Tag HTTP header with noindex. This header works like a meta noindex and will be processed during indexa...

Aug 04, 2022 ⚡ Analysis available

★★ Does the unavailable_after directive really slow down Google's crawling?

The unavailable_after directive leads to a gradual slowdown in crawling over time. Google continues to check the page because it wants to ensure the truthfulness of the information, just as it does wi...

Aug 04, 2022 ⚡ Analysis available

★★ Should you disable Google cache to take control of how your snippets appear in search results?

At the serving level (search results display), it is possible to control the snippets displayed and the cached content of the page. The meta noarchive tag allows you to remove the cached page without ...

Aug 04, 2022 ⚡ Analysis available

★★★ Does Google's removal tool actually delete your URLs from the index?

Google's removal tool masks results at the serving level but doesn't remove the URL from the index. Removal is fast because it happens at search time. For complete index removal, you also need to add ...

Aug 04, 2022 ⚡ Analysis available

★★ Why does it really take Google months to permanently remove a page from its index?

In some cases, it can take several months to completely remove a result from Google's actual index. This is why the removal tool exists: it allows immediate masking at the serving level while permanen...

Aug 04, 2022 ⚡ Analysis available

★★★ Does Google's URL removal tool actually stop crawling your pages?

Using the removal tool doesn't signal Google to stop crawling the page. Google will continue its crawl attempts. To prevent the page from returning to the index, you must combine the tool with noindex...

Aug 04, 2022 ⚡ Analysis available

★★ Could Google start rationing your website's crawl budget for environmental reasons?

Google is increasingly thinking about sustainability and the environmental impact of its crawl and indexation processes, suggesting a potential future evolution in web exploration practices....

Jul 26, 2022 ⚡ Analysis available

★★ Does consistency really beat sporadic effort in SEO?

Consistency is highly important in SEO. It is recommended to maintain a regular and steady routine in content production and monitoring of SEO information sources, rather than trying to do everything ...

Jul 14, 2022 ⚡ Analysis available

★★★ Does robots.txt really prevent your pages from being indexed by Google?

The robots.txt file limits what crawlers can explore on a site, but does not block indexation. If a page becomes very popular with many links, Google can still index the URL without the content, displ...

Jun 30, 2022 ⚡ Analysis available

★★★ Is robots.txt really ineffective at preventing your pages from being indexed by Google?

To reliably prevent a page from being indexed in Google Search, you must use the meta robots 'noindex' tag rather than robots.txt, as the latter is not a foolproof method against indexation....

Jun 30, 2022 ⚡ Analysis available

★★ Can you block indexation of entire directories using server modules instead of robots.txt?

To block indexation of a large portion of a site, you can use Apache modules or Nginx configurations to automatically apply the noindex tag to all URLs under a given prefix or pattern, although this i...

Jun 30, 2022 ⚡ Analysis available

★★ Is indexing your login pages actually hurting your user experience?

Login pages should generally remain indexed because users actively search for them, for example to access their banking portal. Blocking their indexation forces users to navigate unnecessarily through...

Jun 30, 2022 ⚡ Analysis available

★★★ Should you really use rel=canonical over noindex for aging content?

To manage old blog articles that remain relevant, it's better to use the rel=canonical tag pointing to your main page rather than deindexing them with noindex. This allows you to preserve historical a...

Jun 30, 2022 ⚡ Analysis available

★★ Should you really use max-snippet and max-image-preview to control how your content appears in search results?

The 'max-snippet' and 'max-image-preview' meta tags allow you to control the length of text snippets and the size of image previews displayed in search results, with particularly visible effects in Go...

Jun 30, 2022 ⚡ Analysis available

★★★ Is the X-Robots-Tag header really the only way to keep PDFs out of Google's index?

To block indexing of files like PDFs, you must use the HTTP X-Robots-Tag header. If header access isn't available through your CMS, the only alternatives are to not publish the file or use the removal...

Jun 30, 2022 ⚡ Analysis available

★★★ Why does robots.txt actually block images and videos but not web pages?

The robots.txt file works effectively to block images and videos because these contents are indexed in separate tabs (Images, Videos) where Google would have nothing to display as a snippet. For stand...

Jun 30, 2022 ⚡ Analysis available

★★ How does Google really transform your PDFs into searchable content?

When Google indexes a PDF, the first step is to convert it to HTML, then it is processed as standard HTML content for indexing in web results, unlike images and videos which follow distinct indexing p...

Jun 30, 2022 ⚡ Analysis available