Anthropic OpenAI Google Microsoft announce __usage_ai-training robots standard
Anthropic, OpenAI, Google, and Microsoft have announced a new robots.txt standard introducing the token __usage_ai-training to allow webmasters to block or allow all major AI crawlers with two lines of code. However, the author argues that the standard does not grant publishers new control, fails to address how AI companies have already harvested decades of existing content, and serves primarily as a feel-good announcement rather than a practical solution. Open article →