Does the 15 MB Googlebot crawl limit really kill your indexation, and how can you fix it?

What exactly does this 15 MB limit apply to?

Google is talking about raw content (raw bytes), meaning uncompressed data retrieved by Googlebot. Even if your server sends a file compressed with gzip or brotli, Google counts the 15 MB after decompression.

The limit applies resource by resource. Your HTML can weigh 14 MB, your main CSS file 12 MB, and your JavaScript bundle 10 MB — each is treated separately. Googlebot doesn't add up the weights to block the entire page, but it stops dead on any file that exceeds the threshold.

Why does Google impose this restriction?

Crawl budget is an economic reality for Google. Every byte downloaded costs bandwidth, processing time, storage. A strict limit prevents any single file from monopolizing disproportionate resources.

This constraint also protects Google's infrastructure against poorly optimized or malicious sites that might attempt to inject astronomical amounts of content into a single URL. Let's be honest: a 15 MB HTML file is already a symptom of catastrophic architecture.

What happens to content beyond 15 MB?

Googlebot simply stops. It will never see what comes next. If your primary navigation, internal links, or structuring elements are located after this cutoff, they simply don't exist for Google.

Concretely, if your HTML reaches 15.5 MB, the last 500 KB disappear from the radar. This can include entire sections of content, schema.org tags, critical internal linking elements.

• 15 MB of raw content = strict limit per URL, counted after decompression
• Each resource (HTML, CSS, JS, images) has its own independent limit
• Beyond 15 MB, Googlebot stops downloading and ignores the rest of the file
• Critical impact on indexation if essential elements are located after the cutoff
• No cumulative counting: each referenced file restarts its own counter at zero

Is this statement consistent with field observations?

Yes, and it's even an official confirmation of a limit that's been circulating in the SEO community for years. We regularly observe cases where pages with dynamically generated content — e-commerce catalogs, directory listings, infinite category pages — see their HTML explode in size.

The typical problem? Sites that inline massive JSON data, duplicate content in HTML attributes, or load hundreds of products on a single page without pagination. These architectures easily exceed 15 MB and lose entire sections of content in indexation.

What nuances should be applied to this rule?

Martin Splitt doesn't specify whether this limit applies before or after JavaScript rendering. If your initial HTML weighs 2 MB but the DOM generated by your framework reaches 18 MB after hydration, what happens? [To verify] — Google has never publicly clarified this point.

Another gray area: referenced but non-essential resources for rendering. Will a 20 MB background image in a CSS file be partially downloaded or completely ignored? The answer probably depends on context and resource type. We lack concrete data on this scenario.

Finally, this limit is a technical constraint, not a qualitative recommendation. Just because your page weighs 14 MB doesn't mean it's optimized — it remains catastrophic for UX, crawl budget, and Core Web Vitals.

In what cases does this rule really pose a problem?

The most impacted sites are those generating massively aggregated content: vertical search engines, price comparators, marketplaces with hundreds of product variants on a single page.

Concrete observed case: an e-commerce site loading 500 products per category to artificially improve "content depth". Result: 22 MB HTML. Google cut off before the footer, thus losing all critical internal linking elements. Indexation of deep pages collapsed.

How do you verify if your site is affected by this limit?

First step: identify at-risk URLs. Use a crawler like Screaming Frog or Oncrawl and filter pages where HTML exceeds 5 MB — you're already in the red zone. Then examine your JS and CSS resources: a 10 MB webpack bundle isn't uncommon on poorly optimized React applications.

Check with Chrome DevTools the Network tab, "Size" column (uncheck "Disable cache"). Look at the size after decompression, not the transferred size. This is the value that matters to Googlebot.

Also test with curl -H "Accept-Encoding: gzip" -s https://yoursite.com | wc -c to get the raw size that Google retrieves. If the result approaches or exceeds 15,000,000 bytes, you have a problem.

What should you concretely do to stay under this limit?

Paginate. Always. If you display long lists — products, articles, directories — split them into pages of 20-50 elements maximum. Use clean URL parameters (?page=2) with rel="next" and rel="prev" tags if relevant.

Remove all unnecessary content from the HTML source: no redundant JSON-LD, no data duplication between data-* attributes and visible content, no massive inline CSS or JS. Externalize your resources and optimize them individually.

For heavyweight JavaScript applications, adopt code splitting. Load only what's strictly necessary for first render. Lazy-load secondary modules after user interaction. Webpack, Vite, Parcel — all modern bundlers support this approach.

What errors must you absolutely avoid?

Don't attempt to "work around" this limit by artificially splitting your content across multiple URLs without editorial logic. Google detects this type of manipulation and may consider it doorway pages.

Also avoid relying solely on server compression (gzip/brotli) to stay under the limit. It certainly reduces network transfer, but Google counts bytes after decompression. A 25 MB file compressed to 12 MB is still a 25 MB file for Googlebot.

• Audit all pages exceeding 5 MB of raw HTML with a professional crawler
• Measure the actual (non-compressed) size of each resource in Chrome DevTools
• Systematically paginate long lists (products, articles, directories)
• Externalize CSS and JavaScript — never massive inline content in HTML
• Implement code splitting on modern JS applications (React, Vue, Angular)
• Remove all redundancy: unnecessary JSON-LD, duplicated data-* attributes, hidden content
• Regularly test with curl or monitoring tools the raw size of critical resources
• Monitor the evolution of page weight in your monthly crawl budget