Should we stop optimizing for bots and focus solely on the user?

Why does Google refuse to provide specific technical thresholds?

Martin Splitt's stance reflects a deliberate communication strategy. By avoiding the communication of acceptable loading time or CPU consumption ranges, Google protects itself against two major risks.

First, these internal parameters fluctuate with algorithmic evolutions and the crawling infrastructure. Announcing a threshold today could potentially create a false sense of certainty that will become outdated tomorrow. Second, providing a number would encourage "threshold optimization," where sites aim just for the minimum acceptable instead of striving for excellence.

What does Google mean by "optimizing for the user rather than for bots"?

This phrasing, recurrent in official communication, may seem vague. In reality, it is based on a simple principle: a technically efficient site for a human user will also be well crawled by Googlebot.

User-centric metrics—perceived loading time, responsiveness, visual stability—are also relevant constraints for a crawler. A bot attempting to retrieve a page that uses 100% of the CPU for 30 seconds faces the same problem as a visitor on a mid-range mobile: the resource is saturated and the experience is degraded.

Google therefore promotes a holistic approach: if you optimize the Core Web Vitals, reduce blocking JavaScript, compress your assets, and minimize unnecessary requests, you mechanically improve crawlability—without needing to know the exact threshold at which Googlebot abandons.

Does this approach mean that technical details no longer matter?

No. Saying "think user" does not exempt one from mastering the technical aspects of crawl budget, JavaScript rendering, or server resource management. What Google refuses is to provide a binary reading grid of the type "below X ms is good, above is bad".

SEO practitioners must therefore develop a nuanced understanding of performance signals: server response time (TTFB), initial rendering delay, volume of executed JavaScript, number of HTTP requests. Each of these factors influences both user experience and crawl efficiency, but none can be reduced to a single threshold.

• Google will never provide fixed benchmarks for crawling (time, CPU, memory) as these parameters evolve with the infrastructure.
• Optimizing for the user—perceived speed, responsiveness, stability—mechanically improves crawling without needing to know the bot's implementation details.
• The Core Web Vitals (LCP, FID/INP, CLS) serve as relevant proxies: a site that performs well for humans will also perform well for Googlebot.
• The "think user" approach does not exempt one from mastering technical aspects (TTFB, JavaScript, HTTP requests), but it avoids falling into "threshold optimization".
• Practitioners should develop a qualitative reading of performance rather than seeking a magic number that does not officially exist.

Is this statement consistent with what we observe in the field?

Yes, largely. Technical audits show that sites penalized by insufficient crawling almost always exhibit problems with user performance: high TTFB, excessive blocking JavaScript, heavy pages. It is rare for a site that is fast and light for the user to be poorly crawled, except in pathological cases (server errors, poorly calibrated robots.txt blocks, etc.).

However—and this is where Google's discourse becomes strategically vague—there are situations where "bot" optimization diverges from "user" optimization. For example: an e-commerce site with tens of thousands of filtered pages (color, size, price). For the user, these filters are useful. For the crawler, they potentially generate millions of nearly identical URLs that dilute the crawl budget. [To verify]: Google claims to manage these cases via parameter handling and allocated budget, but field experience shows that this remains imperfect.

What nuances should be added to this recommendation?

First, "minimizing CPU resources" is easier said than done on JavaScript-heavy sites. A well-designed SPA (Single Page Application) in React or Vue can deliver an excellent user experience once loaded, but require a significant initial CPU cost during the first render. Google has indeed improved its rendering, but there are still cases where the crawler times out or fails to execute all the JavaScript—especially on deep pages with low internal PageRank.

Second, the recommendation to "think user" assumes that Google crawls from a context comparable to that of an average user. However, Googlebot for mobile operates on a Chromium profile with limited resources, which may not be representative of all devices. A site optimized for an iPhone 14 may not be light for a mid-range Android from 2019—or for the crawler.

Third, there are purely "bot" levers that have no direct impact on the user: log file analysis, optimizing the internal link structure to guide crawling, fine management of noindex/nofollow directives, monitoring HTTP 304 codes to reduce bandwidth. These optimizations do not relate to user experience but influence crawl budget and indexing efficiency.

In what cases does this approach reach its limits?

For very large sites (millions of pages), the correlation "user performance = good crawl" becomes insufficient. One must actively manage crawling via Search Console, monitor crawling rates by page type, identify over-crawled or under-crawled sections, and sometimes restructure the architecture to concentrate the budget on strategic pages.

Likewise, sites with personalized dynamic content (recommendations, geolocation, A/B tests) must ensure that Googlebot sees a consistent and representative version. Optimizing for the user, in this case, may mean displaying different content depending on the context—which is precisely what Google asks to avoid for the crawler (cloaking risk).

What should you do to practically optimize performance and crawl?

Start by measuring the Core Web Vitals on a representative sample of pages (homepage, categories, product pages, articles). Use PageSpeed Insights, Lighthouse, or better yet, the real data from CrUX (Chrome User Experience Report) available in Search Console. These metrics give you a faithful view of what your users experience—and by extension, the complexity faced by Googlebot.

Next, tackle the quick wins: Brotli or Gzip compression, lazy loading of images, CSS/JS minification, aggressive browser caching. These optimizations reduce bandwidth, speed up perceived loading, and lighten CPU load. If your TTFB exceeds 600 ms, investigate on the server side: poorly indexed database, slow SQL queries, lack of object cache (Redis, Memcached).

For JavaScript, prioritize server-side rendering (SSR) or static site generation (SSG) when possible. If you have to use a SPA, ensure that critical content is rendered on the server or pre-rendered, and that deferred JavaScript does not block initial display. Google crawls better than before, but rendering remains costly—limit the number of third-party JS requests and avoid bloated frameworks.

What mistakes should be avoided in this optimization approach?

Don’t fall into the trap of optimization "for the metric." Having an LCP of 1.2 s is pointless if the user sees a blank screen for 3 seconds due to a blocking script. Google values perceived experience, not artificially inflated numbers due to CSS tricks or aggressive lazy loading.

Also avoid neglecting deep pages. Many sites optimize the homepage and main landing pages, then forget level 3 categories, old blog archives, or out-of-stock product pages. However, critical slowdowns often occur there: overloaded templates, unoptimized images, forgotten scripts. An inefficient crawl rarely begins on the homepage.

Finally, do not underestimate the importance of log file analysis. You can have a technically perfect site; if Googlebot spends 80% of its time on unnecessary pages (URL parameters, infinite facets, chained redirects), you lose your crawl budget. Analyze your server logs to understand where the bot is investing its time, and redirect it via robots.txt, canonicals, or strategic noindex directives.

How can I check if my site conforms to this approach?

Start with the Page Experience report in Search Console. It aggregates the Core Web Vitals for mobile and desktop. If you have URLs marked as "Poor" or "Needs Improvement," dive into the details: what types of pages are affected? What are the bottlenecks (LCP, CLS, FID/INP)?

Next, consult the Crawl Stats report. If you notice a drop in the number of pages crawled per day or a rise in the average download time, it’s a signal. Cross-reference this data with your server logs to identify slow pages or 5xx errors that are slowing down the bot.

Also test your key pages with the URL Inspection tool in Search Console. Run a live test to see how Googlebot renders the page, which scripts are executed, and how long rendering takes. If the delay exceeds several seconds, it’s a red flag—even if the page seems fast for a user on a powerful desktop.

• Measure the Core Web Vitals (LCP, INP, CLS) on a representative sample of pages using CrUX or Lighthouse.
• Reduce TTFB to under 600 ms by optimizing server requests, database, and object cache.
• Minimize blocking JavaScript: prefer SSR/SSG, defer non-critical scripts, limit third-party dependencies.
• Analyze server logs to identify over-crawled pages or abnormal download times.
• Use the Page Experience and Crawl Stats reports in Search Console to manage improvements.
• Test strategic pages with the URL Inspection tool to verify Googlebot rendering under real conditions.