Meta · User fetch

meta-externalfetcher

Fetches individual links at a user's request, including for Meta's AI agent features.

Operator
Meta
Type
User fetch · Fetch a page when a person asks an assistant about it. Because a human asked, operators say robots.txt may not apply.
robots.txt token
meta-externalfetcher (case-insensitive)
Follows robots.txt
Not always: it fetches on a user's request
User agent
meta-externalfetcher/1.1
IP ranges
Not published on the operator's crawler page

Verified against Meta’s documentation on November 21, 2026.

Should you block meta-externalfetcher?

Allow it. meta-externalfetcher only visits when a person asks about your page; blocking it means the assistant answers without your content.

If you block it: Meta says this fetcher may bypass robots.txt, because users request the fetch.

robots.txt rules for meta-externalfetcher

A bot with its own group ignores your User-agent: * rules, so copy any Disallow lines you still need (like /cart) into its group.

Allow meta-externalfetcher
User-agent: meta-externalfetcher
Allow: /
Block meta-externalfetcher
User-agent: meta-externalfetcher
Disallow: /

Meta says meta-externalfetcher may not follow robots.txt, because a person triggered the request. To stop it reliably you’d need a firewall rule matching its IP ranges.

How to verify a request is really meta-externalfetcher

Anyone can send a request that claims to be meta-externalfetcher. Meta doesn't publish an IP list on its crawler page, so treat the user agent alone as unverified.

Does your site let meta-externalfetcher in?

The free report checks your robots.txt against the major AI crawlers and tests whether your firewall lets them through.

Other Meta bots

Questions about meta-externalfetcher

What is meta-externalfetcher?

meta-externalfetcher is Meta's user fetch bot. Fetches individual links at a user's request, including for Meta's AI agent features.

Should I block meta-externalfetcher?

Allow it. meta-externalfetcher only visits when a person asks about your page; blocking it means the assistant answers without your content.

What happens if I block meta-externalfetcher?

Meta says this fetcher may bypass robots.txt, because users request the fetch.

← All 20 AI crawlers