Google adds Google-Extended robots token for Gemini training control

Google-Extended is a robots.txt control token rather than a separate web crawler or user-agent, meaning it does not appear in server access logs. Publishers can use it to block Google from utilizing crawled content for training future Gemini models or for grounding AI responses, though Google states that blocking it does not affect normal Google Search inclusion or search rankings. Additionally, Google plans to support the emerging AIPREF shared training preference system in early 2027. Open article →