Google and Bing Back a Shared AI Training Opt-Out Signal for robots.txt
Nicole Wang · Customer Development Manager
Google and Microsoft Bing both moved on October 30, 2026 to let sites state AI training preferences once, for every participating AI provider. Google said it plans to support a shared robots.txt policy token based on an IETF draft, and Bing announced separate controls for AI training and AI grounding. For marketers, the key distinction is training versus citation.
What happened
Google (October 30, 2026). In a Search Central blog post by John Mueller, Google said it aims to support a shared AI training control, together with other AI providers, in 2027. The signal comes from an IETF Internet-Draft from the AIPREF working group. Google said that if a site uses the shared token to opt out of training and does not explicitly name Google-Extended, “Google plans to interpret that as if the publisher had disallowed Google-Extended.” Company-specific controls such as Google-Extended keep working, and sites that allow Google-Extended remain “eligible to appear in responses and receive clicks via grounding in Gemini Apps.”
Microsoft Bing (October 30, 2026). In a post by principal product manager Krishna Madhavan, Bing introduced two independent preferences:
- AI Training: Allow or Disallow, set in robots.txt with the user agent token
__usage_ai-training, or at page level with a meta tag or X-Robots-Tag header. - AI Grounding (page level only): Allow (default), DisallowExtended (Microsoft experiences only, no third-party grounding services), or Disallow.
Bing’s timeline, from the same post:
| Date | Milestone |
|---|---|
| Dec 15, 2026 | Temporary grounding preferences open in Bing Webmaster Tools for sites using NoArchive or NoCache |
| Feb 1, 2027 | New preferences honored as pages are crawled |
| Mar 1, 2027 | NoArchive and NoCache no longer read as training or grounding preferences |
| Sep 30, 2027 | Temporary grounding preferences expire |
Search Engine Roundtable noted that OpenAI and Anthropic have also made related moves, and that the IETF draft is not final. Nate Hake commented that “significant areas of dispute remain.”
Why it matters for B2B marketers
Most B2B SaaS companies want the opposite of what many publishers want: they want to be cited in AI answers about their category. These controls make it possible to separate two decisions that used to be bundled.
- Training is about future models. Blocking it limits whether your content is used to train models. It does not directly control whether you are cited today.
- Grounding is about live answers. Grounding is what Copilot, Gemini and similar tools use to fetch and cite pages in real time. Blocking grounding is what costs visibility.
- Legacy directives will change meaning. If your site uses NoArchive or NoCache (some security and enterprise sites do, for unrelated reasons), Bing will stop reading those as AI preferences after March 1, 2027. You need to set the new preferences explicitly.
What to do now
- Write down a policy. Decide, with legal and content owners, whether you allow training, grounding, both or neither. For most B2B vendors seeking AI visibility, allowing grounding is the priority.
- Audit robots.txt and headers. Check for existing rules on Google-Extended, GPTBot, OAI-SearchBot, ClaudeBot and Bingbot, plus any NoArchive or NoCache directives. Our explainer on robots.txt in SEO covers the basics, and llms.txt vs robots.txt explains where each file fits.
- Do not add the shared token casually. Under Google’s stated plan, using it without a Google-Extended rule is read as disallowing Google-Extended.
- Calendar Bing’s dates. Put December 15, 2026 and March 1, 2027 on your SEO roadmap if you use legacy directives.
- Monitor citations after any change. Track AI answer presence before and after edits so you can see if a rule change cost you visibility. See how AI engines cite sources.
Sources
- Google Search Central blog, AI training control (Oct 30, 2026): https://developers.google.com/search/blog/2026/10/ai-training-control
- Microsoft Bing blog, “Your content, your choice: clearer preferences for training and grounding” (Oct 30, 2026): https://blogs.bing.com/search/2026/10/your-content-your-choice-clearer-preferences-for-training-and-grounding/
- IETF draft, AI training preference signal: https://datatracker.ietf.org/doc/draft-kelley-aipref-training-preference-signal/
- Search Engine Roundtable, Google adopts AI training control (Oct 30, 2026): https://www.seroundtable.com/google-ai-training-control-42430.html
- Search Engine Roundtable, Bing AI controls (Oct 30, 2026): https://www.seroundtable.com/microsoft-bing-ai-controls-42429.html
Frequently asked questions
What is the shared AI training signal?
A robots.txt policy token based on an IETF AIPREF draft that lets a site express one AI training preference for all participating providers. Bing documents it as the user agent token __usage_ai-training.
Does opting out of AI training remove my site from AI answers?
Not by itself. Bing separates training from grounding, and Google says sites that allow Google-Extended stay eligible for grounding in Gemini Apps. Blocking grounding is what reduces AI answer visibility.
