Googlebot
Crawls for Google Search, including all Search features (such as AI Overviews), Discover, Images and News.
- Operator
- Type
- AI search · Index pages so an assistant can find and cite them in answers. Blocking these is how sites disappear from AI search.
- robots.txt token
Googlebot(case-insensitive)- Follows robots.txt
- Yes
- User agent
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Verified against Google’s documentation on November 21, 2026.
Should you block Googlebot?
Allow it. Googlebot is how Google finds pages to show and cite. For an online store there's rarely a good reason to block it.
If you block it: Your pages drop out of Google Search, and with it Google's AI features built on Search.
robots.txt rules for Googlebot
A bot with its own group ignores your User-agent: * rules, so copy any Disallow lines you still need (like /cart) into its group.
User-agent: Googlebot
Allow: /User-agent: Googlebot
Disallow: /How to verify a request is really Googlebot
Anyone can send a request that claims to be Googlebot. Google publishes the IP ranges it uses: match the request's IP address against that list before trusting it, and before allowing it through a firewall.
Does your site let Googlebot in?
The free report checks your robots.txt against the major AI crawlers and tests whether your firewall lets them through.
Other Google bots
Questions about Googlebot
What is Googlebot?
Googlebot is Google's ai search bot. Crawls for Google Search, including all Search features (such as AI Overviews), Discover, Images and News.
Should I block Googlebot?
Allow it. Googlebot is how Google finds pages to show and cite. For an online store there's rarely a good reason to block it.
What happens if I block Googlebot?
Your pages drop out of Google Search, and with it Google's AI features built on Search.