Is it possible for Googlebot Desktop crawling to cause indexing delays?
Google’s John Mueller responds to a query regarding whether it mattered if a website is crawled by Googlebot mobile or Googlebot desktop. The individual who asked the inquiry was concerned about sluggish indexing and wanted to know whether it was because they were crawled by Googlebot desktop (rather than Googlebot mobile).
Google employs a slew of internet crawlers that hop from link to link in search of new web sites to scan and index.
If you examine your traffic records, you may notice:
- Googlebot Desktop
- Googlebot Smartphone
- Googlebot Image
- Mediapartners-Google (Mobile AdSense)
- FeedFetcher-Google
- Google-Read-Aloud
- AdsBot-Google
- Googlebot-Video
These are only a handful of the Google crawlers that can visit a website.
The question John Mueller answered was largely on Googlebot Desktop and Googlebot Smartphone, and what influence the desktop version may have on indexing.
This is the question:
“Our website’s posts are being indexed after one week. The site is old and mobile-friendly.
When I submit URLs in Inspection Tools, it shows it’s crawled by Google smartphone.
But in the About section, the indexing crawler is shown as Googlebot Desktop.
Does this mean the Smartphone bot is crawling the URL and the Desktop bot is indexing this URL?
Is this the reason behind the indexing delay?”
These are just a handful of the Google-related Google crawlers that can visit a website.
The question John Mueller answered was largely on Googlebot Desktop and Googlebot Smartphone, and what effect the desktop version may have on indexing.
“This would not be a reason for any kind of indexing issues.
I don’t know offhand why you might see this mix of desktop and mobile crawlers there.
It might just be that the Search Console is essentially saying well, by default, the Inspection Tool will just use the mobile Googlebot.
In practice, we always crawl with both of these crawlers.
It’s just a matter of how much we crawl and index with any of these crawlers.
So, usually you’ll see something like 80% is coming from mobile Googlebot and 20% maybe from desktop Googlebot.
And that kind of ratio, that is essentially normal and doesn’t mean that things are slower than anything else.”
Several Versions of Googlebot Crawls Websites
According to Google’s official:
“Your website will probably be crawled by both Googlebot Desktop and Googlebot Smartphone.
You can identify the subtype of Googlebot by looking at the user agent string in the request.”
According to Mueller and Google, being crawled by both is completely normal, and there is no indexing harm associated with being crawled by Googlebot Desktop.
