Can a single line of robots.txt really control AI? Unpacking the new agreements from Google, Microsoft, and OpenAI.
By Nokka (นก-กา) | 3 Nov 2026 On 30 October 2026, three major companies made the same announcement on the same day. Google, Microsoft, and OpenAI said they would obey the middle line, one line in robots.txt, that says a website does not allow its content to be used to train AI.[1][2][3] If this can actually be done, website owners will not have to write separate rules for each company as they do today. But the "if" in the sentence above is the crux of the matter. What exactly is this agreement? It starts with a draft that…
Google, Microsoft, and OpenAI have announced that they will respect a new line in the robots.txt file that allows website owners to opt out of having their content used for AI training. This line, called the "policy token," would enable owners to specify whether their content can be used for AI training, without having to write separate rules for each company.
The companies have different implementation plans, with Google, Microsoft, and OpenAI expected to start supporting the new line in the coming months, with Microsoft to start from February 1, 2027.
Written by urgent.news from Dev.to's report — not a translation of it. Machine-written — may contain errors; check the original before relying on it.
