🔍 Read the full analysis: One Opt-Out For AI Training: How To Set It And What To Expect on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will support a shared robots.txt signal asking that a website’s content not be used to train generative AI models. The signal is voluntary, does not block crawling or AI answers, and will take effect on different schedules; Meta has voiced support but has not committed to honoring it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared robots.txt signal asking that website content not be used to train generative AI models. The proposal could spare publishers from maintaining separate training opt-outs for each participating company, but it is a voluntary request, not a technical block, and the companies plan to implement it on different timelines.
The proposed rule is a group in a website’s robots.txt file: User-Agent: __usage_ai-training followed by Disallow: /. It comes from an individual Internet-Draft titled Machine-Readable Training Preference Signal, published October 30 by Anthropic standards lead Kevin Kelley. The rule expresses a publisher’s preference against using the site’s content for generative AI training.
Anthropic says it has started implementation, though its crawler help page still described company-specific controls as of October 31. OpenAI says it aims to implement the signal “in the coming months.” Microsoft says it will honor it from February 1, 2027, as it crawls websites. Google’s documentation gives a target of early 2027. Meta’s communications chief expressed support for a common signal but did not say whether Meta will honor it.
Microsoft also announced a separate page-level preference for AI Grounding—the use of web content to inform answers in Bing, Copilot and Microsoft 365. Microsoft says both preferences take effect February 1, 2027, and that its existing NoArchive and NoCache tags will stop counting as training or grounding preferences after March 1, 2027. Those changes are separate from the shared training signal.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A Simpler Signal, With Limits
For publishers, a common rule could reduce the work of expressing a training preference across multiple AI companies. A site operator can add the shared entry while continuing to use existing company-specific controls until each provider supports the new signal. That distinction matters: a shared rule does not take effect everywhere at once, and the participating companies’ implementation plans remain staggered.
The signal does not itself prevent a crawler from accessing a site. It communicates a preference that companies say they will honor, rather than enforcing a restriction through the web’s technical infrastructure. It also concerns model training, not every use of web content. AI-generated answers based on live or retrieved web pages may have separate controls; Microsoft’s grounding preference is one example. The signal does not remove material from models already trained on it or establish a right to payment.
Microsoft has said that its controls do not guarantee inclusion, traffic or payment. It also says the new preferences do not change eligibility for Bing search results. Google says Google-Extended does not affect Google Search. For site owners, the practical point is to decide separately what rules to set for training, AI answer services and search, rather than assuming one opt-out covers all three.
Products Worth Considering
Samsung 27-Inch Odyssey QD-OLED G8 4K 240Hz Gaming Monitor
Samsung 27 Odyssey OLED G5 (G50SF) QHD QD-OLED Gaming Monitor
LG UltraGear 32GX850A-B 32-Inch 4K UHD Glossy OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
From Company Rules to One Draft
Before this proposal, publishers could use different controls for different services, including GPTBot for OpenAI, Google-Extended for Google and ClaudeBot for Anthropic. Microsoft’s NoArchive and NoCache tags also had specific roles. A publisher that wanted to express a preference across providers needed to know which controls each company recognized and maintain them accordingly.
The shared signal is being discussed through the IETF’s AI Preferences working group. It is not yet an approved internet standard: the document is an individual Internet-Draft, and the source material says it has no formal standing in the IETF standards process. At a working-group meeting on October 21, one of the chairs said, “We haven’t declared consensus on anything yet.” Company support for the draft should not be confused with formal standards approval.
The proposal’s authorship and the reporting behind it also warrant clear attribution. The draft was written by Anthropic’s Kevin Kelley. ThorstenMeyerAI.com disclosed that the article informing this account was researched and drafted with AI assistance, including Anthropic’s Claude, and that the publisher belongs to a network of websites affected by AI companies’ use of web content.
““has already started implementing support for this draft””
— Anthropic, describing its implementation
Coverage and Enforcement Remain Open
Meta has not made an implementation commitment, and the source material reports no statement from Apple, Amazon, xAI or Perplexity about whether they will honor the signal. Support from the four named companies therefore does not establish that the rule will be recognized across the AI industry.
It is also not yet clear how consistently the voluntary signal will be applied across each company’s services, or how publishers could address a company that ignores it. The draft’s status in the IETF process remains unsettled. The signal’s stated scope is training, so publishers should not treat it as a ban on crawling, a control for every AI-generated answer, a way to erase previously trained material or a means of securing payment.
Watch Rollouts Through 2027
The next test is whether each company follows through on its stated schedule and makes the rule available across its services. Anthropic says implementation has begun; OpenAI has given a broad target of the coming months; Google’s documentation points to early 2027; and Microsoft’s stated date is February 1, 2027. Publishers will also be watching for any change in Meta’s position and announcements from other AI companies.
For now, site operators considering the opt-out can add the shared rule while keeping existing company-specific entries in place until providers confirm support. Microsoft users of NoArchive or NoCache should review Microsoft’s separate dates, including the March 1, 2027 change to those tags’ role as training or grounding preferences. The draft’s progress through the IETF working group is another milestone; until then, the signal remains a proposed preference supported by company announcements, not an adopted internet standard.
Key Questions
The proposed robots.txt group is User-Agent: __usage_ai-training followed by Disallow: /. It expresses a preference against using the site’s content to train generative AI models.
Does the rule stop AI companies from crawling a website?
No. The signal communicates a training preference; it is not a technical block on crawling. Its effect depends on companies choosing to honor it.
Does opting out of training also block AI answers using the site?
Not under the shared signal as described. It covers training, while use of web content to inform AI answers may have separate controls. Microsoft announced a distinct AI Grounding preference for certain services.
When will the companies support the signal?
Anthropic says implementation has started, OpenAI aims for the coming months, Google’s target is early 2027, and Microsoft says February 1, 2027. The rollout schedule differs by company.
Is this an official internet standard, and has Meta committed to it?
Neither is established by the announcements. The signal is in an individual Internet-Draft and has not been adopted as an IETF standard. Meta’s spokesperson expressed support for the idea but did not commit the company to honoring it.
Source: ThorstenMeyerAI.com
