What has changed for website owners?
On 30 October 2026 OpenAI, Google and Microsoft announced on their blogs that they will honour a shared signal in robots.txt. That is the text file that tells automated programs what they may read on a site. The new signal says one thing only: whether your text may be used to train AI models.
Until now you had to write a rule for each company, one for OpenAI's GPTBot, one for Google-Extended and so on. With the shared signal you write one rule, and every participating company reads it. It comes from a working group at the IETF, the body that writes internet standards. For now it is a draft, not yet a standard.
Which lines do you need to add?
Microsoft's blog gives the example. To opt your whole site out of training, add these two lines to robots.txt.
- User-Agent: __usage_ai-training
- Disallow: /
To close only part of the site, put the folder in place of the slash, for example /price-list/. If you write nothing, the answer is yes and training is allowed. The signal covers how your text is used, not whether the site is read, so it does not remove you from search results.
When does it apply, and to whom?
- Microsoft honours it from 1 February 2027 for Bing, Copilot and Microsoft 365, as it recrawls pages.
- Google plans to support it in 2027. An opt-out through the shared signal will count as an opt-out from Google-Extended.
- OpenAI says it will apply it in the coming months. Its GPTBot rules, for the crawler that collects training text, stay in force.
If you use NoArchive or NoCache for Bing today, there is a deadline. After 1 March 2027 Microsoft will no longer read them as a preference on training or on use in AI answers. You need to publish the new signal before that date.
Should you opt out?
It depends on what your site sells. Take a publisher of technical guides that lives on subscriptions, or a company whose price list took years to build. There, opting out of training protects something of value. An agency that publishes articles to be found has fewer reasons to close the door.
Keep two things apart. Opting out of training is not the same as blocking the programs that look up answers. According to OpenAI, sites that block OAI-SearchBot do not appear in ChatGPT search answers. Microsoft warns that refusing use in AI answers reduces your brand's visibility in those services.
What about us?
On our own site we made the opposite choice. Our robots.txt lets GPTBot, ClaudeBot, PerplexityBot and Google-Extended read the public pages and closes only the technical folders. We write these articles partly so that AI assistants read and cite them.
So what should you do?
- If your site sells content, such as guides, product sheets or price lists, add the two shared-signal lines and keep your existing GPTBot and Google-Extended rules.
- If your site exists to be found, do not block OAI-SearchBot or search engines, and take your time on training.
- If you use NoArchive or NoCache for Bing, put 1 March 2027 in your calendar.
“Robots.txt is where you decide who may learn from your work. Read it again once a year.”
DBrothers — on website rules for AI
Frequently asked questions
Will the shared signal remove my site from Google?
No. It covers the use of your text for training, not your place in search results. Microsoft also says it changes nothing for Bing search results.
Do my existing GPTBot rules still apply?
Yes. OpenAI and Google say that GPTBot and Google-Extended keep working alongside the shared signal. Bing is different, because its new preferences replace NoArchive and NoCache.
Does opting out cover text already collected?
The announcements describe how the companies will use content once they read the signal. They say nothing about text already collected, so do not assume it.
Open your site's robots.txt today and check which rules it has for AI. If you do not know who wrote it or what it says, it is the first file to review with whoever runs your site.
