Anthropic · Training

ClaudeBot

Collects public web content that may be used to train and improve Anthropic's models.

Operator
Anthropic
Type
Training · Collect content that may be used to train AI models. Blocking them doesn't remove you from AI search answers.
robots.txt token
ClaudeBot (case-insensitive)
Follows robots.txt
Yes
User agent
Anthropic publishes the robots.txt token, not a full user agent string. Requests contain "ClaudeBot".

Verified against Anthropic’s documentation on November 21, 2026.

Should you block ClaudeBot?

Your call. Blocking ClaudeBot keeps your content out of future training data and doesn't remove you from AI search answers. Many stores allow it for brand familiarity; many publishers block it.

If you block it: Your future content is excluded from Anthropic's training datasets. It doesn't affect Claude-SearchBot or Claude-User.

robots.txt rules for ClaudeBot

A bot with its own group ignores your User-agent: * rules, so copy any Disallow lines you still need (like /cart) into its group.

Allow ClaudeBot
User-agent: ClaudeBot
Allow: /
Block ClaudeBot
User-agent: ClaudeBot
Disallow: /

How to verify a request is really ClaudeBot

Anyone can send a request that claims to be ClaudeBot. Anthropic publishes the IP ranges it uses: match the request's IP address against that list before trusting it, and before allowing it through a firewall.

Does your site let ClaudeBot in?

The free report checks your robots.txt against the major AI crawlers and tests whether your firewall lets them through.

Other Anthropic bots

Questions about ClaudeBot

What is ClaudeBot?

ClaudeBot is Anthropic's training bot. Collects public web content that may be used to train and improve Anthropic's models.

Should I block ClaudeBot?

Your call. Blocking ClaudeBot keeps your content out of future training data and doesn't remove you from AI search answers. Many stores allow it for brand familiarity; many publishers block it.

What happens if I block ClaudeBot?

Your future content is excluded from Anthropic's training datasets. It doesn't affect Claude-SearchBot or Claude-User.

← All 20 AI crawlers