Everyone should slow down AI development except for me
In an attempt to curb AI companies from aggressively scraping websites, administrators have implemented the Anubis system. This system uses a Proof-of-Work scheme similar to Hashcash, originally designed to reduce email spam. While the additional load on individual servers is considered negligible, at a large scale, it significantly increases the cost of scraping.
The system serves as a temporary solution, allowing for more time to focus on identifying and blocking headless browsers, which are often used by AI crawlers. Users should be aware that Anubis requires modern JavaScript features that plugins like JShelter will disable, thus disabling JShelter or similar plugins for this domain is necessary to access the website.
Written by urgent.news from Lobsters's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
This story
This is one outlet's version. Read the fullest account.
