Meta Walked Back Its AI-Training Terms. What Website Owners Must Do Before October 30

TL;DR — Meta's new commercial terms take effect October 30, 2026. After a publisher revolt, Meta removed the word "train" from the text — but the crawling permission stays, and acceptance is passive: keep your Page or ads running and you've agreed. The one practical move before the deadline: block meta-externalagent in robots.txt (copy-paste code below). It won't break your link previews, but it also isn't a contract — know the limits.

Disclosure: this article contains no affiliate links. We verified every claim below against Meta's documentation and reporting from October 15–20, 2026, linked in-line.

In mid-October, Meta published updated Commercial Terms that would let the company use automated software — and "test credentials" — to access, index, cache, copy, and re-crawl websites linked to any business using its products, and use that content to "provide, develop, improve, research, train, and maintain" its services. Nieman Lab reporters Andrew Deck and Hanaa' Tameez first described the change on October 15. The effective date: October 30, 2026.

Within 48 hours, publishers, creators, and media lawyers revolted. Meta blinked — twice. Here's what actually changed, what didn't, and the five-minute robots.txt fix every website owner should apply before October 30.

The Clause That Started It

The original preview language was remarkably broad. As reported by PPC Land and Martin Cid Magazine, the terms covered anyone using Meta's tools for business: Pages, Groups, ads, marketing messages, product catalogues, the Business Suite dashboard. The Register noted at least a million businesses use Meta's AI agents in WhatsApp and Messenger alone. A bakery selling through Instagram faced the same choice as a national newspaper.

The clause also included a "test credentials" provision — meaning Meta could bypass registration walls. Gated content, developer docs, and paid tiers were all arguably in scope, as ByteIota flagged. And there was no opt-out: continued use of Meta's business products after October 30 counts as agreement.

The backlash was swift and specific. Law professor James Grimmelmann called the clause "absurdly broad." Media analyst Joshua Kint argued it pushed Meta outside its walled gardens. The News/Media Alliance's Danielle Coffey put "millions, or billions" of dollars in licensing value at stake. UW law professor Benjamin Sobel compared the setup to what antitrust law calls "tying": to keep the business tools, you have to accept the scraping.

The Two-Stage Walkback

Meta retreated in two steps over October 16–17:

1. Denial. Meta spokesperson Andy Stone said on October 16 that the clause was narrower than critics suggested — designed, he wrote, to pull accurate product information for on-platform ad and content recommendations, "not about scraping news sites."
2. Revision. The preview text was then revised to drop the word "train" from the listed purposes. Per rrtechpulse and Readus247 (status checked October 19), the crawl clause now names advertising, business products, and security as its purposes — and Meta says "content owners can use robots.txt to signal their preferences" and that the updates "don't give us new rights that supersede those preferences." That statement is Meta's position, not wording in the terms.

What didn't change: the crawling permission itself, the test-credentials language, the October 30 effective date, the passive acceptance, and the absence of robots.txt from the contract. As Readus247's October 19 status check puts it: the live text carries no crawling clause today; the disputed language belongs to the version that starts October 30. Meta has published no redline and no regulator has opened a public action.

The 5-Minute Fix: Block the Training Crawler in robots.txt

Meta's own crawler documentation gives site owners the practical lever: name the crawler you want to block in your robots.txt file, which lives at https://yoursite.com/robots.txt. The four Meta user agents you should know:

User agent	What it does
meta-externalagent	Crawls for uses such as training foundation AI models or improving products by indexing content. This is the one to block if you don't want your content feeding models.
meta-externalads	Crawls to improve advertising and other business products. Block it too if you don't want Meta indexing your site for ad products.
meta-webindexer	Builds the index behind Meta AI search answers. Meta says allowing it helps it cite and link to your content — keep it allowed if you want visibility in Meta AI.
facebookexternalhit	Creates link previews when someone shares your URL. Leave it alone — blocking it breaks your preview cards.

The copy-paste block. Add this to the top of your robots.txt:

User-agent: meta-externalagent
Disallow: /

User-agent: meta-externalads
Disallow: /

This blocks both the training crawler and the ads crawler site-wide, while leaving link previews (facebookexternalhit) and Meta AI citations (meta-webindexer) working. Per Meta, changes can take up to 24 hours to apply.

Pro tip: on WordPress, you can edit robots.txt with an SEO plugin (Rank Math and Yoast both have a robots.txt editor) or place a physical file at your site root. On static hosts, drop the file in your publish directory and redeploy. After saving, verify at yoursite.com/robots.txt in an incognito window.

What robots.txt Cannot Do (Be Honest With Yourself)

This is where honest coverage matters more than comforting coverage. The block above is worth doing — it costs nothing, and Meta says it respects those preferences — but it has real limits:

• It's not a contract. Meta's terms do not mention robots.txt, and as the News/Media Alliance notes, the terms do not promise Meta will honor it. The preference lives in Meta's documentation, not in the agreement.
• Two crawlers may bypass it. Meta's own docs state that meta-externalfetcher — which fetches pages when a user shares or requests a link — may bypass robots.txt, and facebookexternalhit may bypass it during security or integrity checks.
• One token, two purposes. PPC Land notes that meta-externalagent is a mixed-use agent: the same robots.txt token covers training and indexing. You can't block model training while allowing product-indexing through this token alone.
• The only true opt-out is exit. The terms' only described exit is to stop using Meta's business products before October 30 — giving up your Page, ads, and Business Suite tools. For most businesses, that's not realistic.

If your website content is what you sell — subscriptions, licensed photography, proprietary data — a robots.txt line is a first step, not a strategy. Consider legal advice about a written carve-out before October 30.

Should You Block, or Stay Visible?

It's a genuine trade-off, and it depends on your business model:

• Block meta-externalagent if your content is the product — paywalled articles, stock media, courses, proprietary research. The licensing angle is real: the News/Media Alliance framed it as "millions, or billions" at stake, and giving training rights away for free weakens every future negotiation.
• Keep meta-webindexer allowed if you want to be cited in Meta AI answers. Meta AI search is another answer engine building its own index — and like Google, it sends traffic to the sources it cites. Being discoverable there is the same logic as staying indexable for search.
• Check your server logs. Before deciding, look at which Meta bots actually visit your site. If meta-externalagent barely touches your pages, the block is pure insurance. If it crawls you heavily, you've just confirmed why the block matters.
• Keep a dated record. Note the date you added the block and screenshot your robots.txt. If crawler behavior ever becomes a legal question, your contemporaneous record is evidence.

The Bigger Picture

This fight is part of a larger 2026 pattern: AI companies writing crawler permissions into terms, defaults, and protocols while the people asked to give permission say yes without meaning it. Google ran a contribution pilot where payment is optional and use is not. Eight US states are suing OpenAI over ChatGPT risks. Publishers won this round in 48 hours — the fastest reversal of its kind — precisely because they named the clause, quoted the language, and moved together.

The next test is whether Meta's stated respect for robots.txt preferences survives contact with the October 30 terms, which don't commit to it in writing. October 30 is the date to watch. Set a calendar reminder for October 31 and check your server logs.

For related reading: our guide on what happens when AI agents get blocked from websites covers the technical side of crawler control, and our coverage of the Anthropic AI safety bill debate covers the policy side of the same fight over who sets the rules for AI.

Frequently Asked Questions

Did Meta cancel the new commercial terms?

No. Meta's updated commercial terms still take effect on October 30, 2026. After publisher backlash, Meta removed the word "train" from the listed purposes in the preview text, but the crawling permission — automated access, copying, and re-crawling of websites tied to your business products — remains.

Which Meta crawler trains AI models?

Meta's developer documentation names meta-externalagent/1.1 as the crawler that collects web content for uses such as training foundation AI models or improving products by indexing content. Blocking that exact user agent in robots.txt is the practical lever site owners have before October 30.

How do I block Meta's AI training crawler?

Add two lines to the robots.txt file at your site's root: User-agent: meta-externalagent followed by Disallow: /. Meta says crawler changes can take up to 24 hours to apply.

Will blocking meta-externalagent hurt my Facebook posts?

No. Link previews come from facebookexternalhit and Meta AI citations come from meta-webindexer — different crawlers. Blocking only meta-externalagent leaves both working, so your shared posts keep their preview cards.

Does robots.txt override Meta's terms?

No. It's a request that well-behaved crawlers follow, and Meta says it respects those preferences, but the contract doesn't commit to honoring it. Meta's docs also note that meta-externalfetcher and facebookexternalhit may bypass robots.txt in certain cases.

Do I have to agree to Meta's new terms?

There's no separate consent step. Businesses accept the terms by continuing to use Meta's commercial products — Pages, ads, Business Suite, WhatsApp Business — after October 30, 2026. The only exit the terms describe is to stop using Meta's business products.