Loading page
Robots.txt control token letting publishers decide whether Google may use crawled content to train and ground its Gemini AI models.
Google-Extended is Google's robots.txt control token for managing whether your content can be used to train and ground its generative AI models, including Gemini. It is not a separate crawler: crawling is still done by Googlebot, and Google-Extended controls how the crawled content may be used. Site owners can block Google-Extended in robots.txt without affecting their traditional SEO rankings. This gives publishers control: allow Google Search indexing while blocking AI training, or allow both for maximum visibility.
Qwairy checks whether your robots.txt allows Google-Extended as part of its crawlability analysis. Know at a glance whether your content can be used to train and ground Google's Gemini models, alongside the rest of your AI crawler directives.
Automated web client operated by an AI company for model training, AI search, or user-requested retrieval.
Text file placed at the root of a website to indicate to indexing robots which pages to explore or avoid.
Family of language models developed by Google, integrated into its search and AI products.