The decision behind a public portfolio
A photographer wants the portfolio found. A production company wants its work considered for the next job. Deciding how AI services may use those pages calls for a more specific conversation with the webmaster than a request to block bots.
On October 30, 2026, Google announced plans to support a shared AI training control in robots.txt, aiming for 2027. Google's crawler documentation puts the target at early 2027. The proposal would let websites express training preferences across participating AI providers in one step, while company specific controls keep working alongside it.
For Florida owners and creators, and for site owners anywhere, the useful work now is to decide what they want public, how they want it discovered and which AI uses they want to permit. Then ask the person maintaining the site to match those choices to the right controls.
What the proposed shared signal would do
The Internet-Draft linked by Google describes a policy token, written as __usage_ai-training in a robots.txt User-Agent line, for expressing training preferences for particular website paths. It remains a work in progress, not a finished standard. The draft explicitly separates content use preferences from crawler traffic rules: the training token itself does not instruct a crawler whether to fetch a page.
One detail matters for anyone with an existing robots.txt file. Under the draft, a general User-Agent: * rule does not count as a training preference. An owner who wants to state one would need the new token itself.
Google's announced approach has an important condition. If a site uses the shared signal to opt out of training and does not explicitly specify Google-Extended, Google plans to treat that preference as if the site had disallowed Google-Extended. Owners should therefore ask how existing provider specific choices would interact with a future shared rule.
What Google-Extended already controls
Google's crawler documentation says Google-Extended manages whether content Google crawls can be used to train future generations of Gemini models and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI. Grounding means supplying content from the Google Search index to the model at prompt time to improve factuality and relevance.
The same documentation says Google-Extended does not affect a site's inclusion in Google Search and is not used as a Search ranking signal. It is a control token, not a separate crawler identity that owners should expect to see in request logs.
The trade off runs both ways. Google's announcement notes that websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps. Before approving a change, ask the webmaster to explain both training and grounding consequences. Calling it only a training switch would leave part of that decision out.
Google Search has separate choices
Google's Search content controls documentation lists a separate, sitewide Search generative AI control in Search Console for managing inclusion in generative AI features on Google Search, including AI Overviews and AI Mode. It also describes page and text level settings for search presentation, and the noindex rule for keeping content out of Search results.
Start by naming the outcome you want. Limiting Gemini use, changing search previews and excluding a page from Search require different discussions. Ask the maintainer to identify the setting, affected pages and expected effect before changing anything.
Give the webmaster a clear work order
Request a copy of the current robots.txt file and a record of relevant CMS and Search Console settings. List which pages should help customers discover the business and which material should remain private. Ask for a proposed change and a way to check its result.
Google's robots.txt guidance warns that these instructions cannot enforce crawler behavior; it is up to each crawler to obey them. A page URL blocked in robots.txt can still appear in Search if it is linked from other places on the web. For confidential files, Google recommends stronger methods such as password protection.
Keep a record of the owner's choices and who approved the implementation. That gives the next website contractor something more useful than an unexplained collection of bot rules.
Sources
- Google's October 30 announcement
- Training preference Internet-Draft, work in progress
- Google-Extended documentation
- Google Search content controls
- Robots.txt uses and limits
Key Facts for Owners and Search
When does Google plan to support the shared signal?
Google's October 30 announcement targets 2027, and its crawler documentation says early 2027. It describes a plan, not a completed rollout.
Does Google-Extended change Google Search rankings?
No. Google says it does not affect a site's inclusion in Google Search and is not used as a ranking signal.
Does Google-Extended cover more than training?
Yes. Its documented scope also includes grounding in Gemini Apps and Grounding with Google Search on Vertex AI.
Is there a separate control for Google's AI search features?
Yes. Google lists a Search generative AI control in Search Console for managing sitewide inclusion in AI Overviews and AI Mode.
Can robots.txt keep private files secure?
No. Its rules cannot enforce crawler behavior. Google recommends password protection for private files.
