NEWv1.17: Audited & Actionable
Qwairy
Technical

Google-Extended

Robots.txt control token letting publishers decide whether Google may use crawled content to train and ground its Gemini AI models.

What is Google-Extended?

Google-Extended is Google's robots.txt control token for managing whether your content can be used to train and ground its generative AI models, including Gemini. It is not a separate crawler: crawling is still done by Googlebot, and Google-Extended controls how the crawled content may be used. Site owners can block Google-Extended in robots.txt without affecting their traditional SEO rankings. This gives publishers control: allow Google Search indexing while blocking AI training, or allow both for maximum visibility.

How Qwairy Makes This Actionable

Qwairy checks whether your robots.txt allows Google-Extended as part of its crawlability analysis. Know at a glance whether your content can be used to train and ground Google's Gemini models, alongside the rest of your AI crawler directives.

Frequently Asked Questions

No, blocking Google-Extended has zero impact on Google Search rankings. Google-Extended is only for AI training and grounding (Gemini), while Googlebot handles traditional search indexing. You can block Google-Extended while allowing Googlebot if you want search visibility without AI training. However, blocking Google-Extended means your content won't be used to train or ground Gemini responses.

To give publishers control. Many publishers wanted to prevent AI training while maintaining search rankings. Google-Extended lets you choose: 'User-agent: Google-Extended / Disallow: /' blocks AI training only. This addresses concerns about AI companies using content without permission. However, blocking also means your content won't be used to ground Gemini responses.

Yes, if you want your content used in Gemini responses. Blocking Google-Extended means your content won't be used to train or ground Google's Gemini models. Note that AI Overviews are governed by standard Google Search indexing (Googlebot), not Google-Extended, so blocking it does not remove you from AI Overviews. Only block if you have specific licensing concerns or want to negotiate commercial deals.
Share:

Related Terms

AI Crawler

Technical

Indexing robot used by AI companies to collect data intended to train or feed their models.

robots.txt

Technical

Text file placed at the root of a website to indicate to indexing robots which pages to explore or avoid.

Gemini

AI Platforms

Family of language models developed by Google, integrated into its search and AI products.

Previous
GEO Audit

Comprehensive evaluation of a website's AI search visibility and optimization opportunities.

Next
GPTBot

OpenAI's web crawler that collects public web content to train future GPT models; site owners control its access via robots.txt.

← Back to GlossaryLast updated Jun 9, 2026