The 4 AI Giants Just Agreed to Let Websites Opt Out. They’ve Already Taken the Data
Agreeing to honor a single robots.txt signal doesn’t solve the bigger problem of scraping the entire internet already.
We haven't written up this one. Inc. has the full story — the link below goes straight to it.