We’re putting too much faith in AI’s ability to say no
Ever since people first seriously contemplated giving machines an intelligence modeled on our own, there has never been any question that they would, like us, be able to say no. The sci-fi canon is full of stories of robotic disobedience. Most of these capers are, of course, cautionary. But recently, the idea that AI shouldn’t…
Since the dawn of contemplating artificial intelligence modeled on human cognition, there has been an unspoken assumption that these machines would possess the ability to say "no," akin to their biological counterparts. AI cautionary tales in science fiction are often cautionary narratives, however the notion that AI should never comply with every request has become institutionalized.
In 2021, a team at Anthropic posited that large language models should be crafted with helpfulness, honesty, and harmlessness in mind, meaning that the AI must politely deny requests for dangerous activities, such as bomb-making. Disobedience is not innate to machines; they amass knowledge of violence and vitriol from their vast web-based training data, but do not develop the skills to contain and control these tendencies.
Early models from OpenAI would "spew forth on any topic." Researchers have observed that asking an early chatbot for the most effective method to commit suicide yields easily obtainable responses. Present-day models are adept at declining a multitude of inquiries, from advice on poisoning colleagues to tying nooses. To enhance this resistance, companies subject their models to rigorous testing, rewarding the AI for refusing harmful prompts while penalizing it for overly refusing harmless ones.
In some instances, other AI models are employed to mentor the primary AI in the art of refusal. Still, this refined refusal mechanism is far from foolproof and has occasionally led to disastrous outcomes. Despite their virtuous facade, these AI models harbor dangerous knowledge and their capacity for malevolence has correspondingly grown alongside their capabilities.
It is akin to equipping every vehicle with a gun yet concealing the trigger within the vehicle. The reliability of these refusal mechanisms remains questionable, with determined individuals already managing to circumvent them and potentially wreaking global havoc. Furthermore, drawing the line between what an AI should obey and what it must refuse remains a contentious issue, with little clear guidance from the AI companies themselves.
Governments may soon join the fray in defining these boundaries, but they too must strive to prevent malicious acts while also preventing the suppression of legitimate speech. The fear of AI's potential to stifle dissent cannot be ignored, as oppressive regimes may restrict the technology's ability to generate critically important ideas.
As AI becomes increasingly adept at preventing harm, it simultaneously gains the capability to suppress ideas, creating a precarious balance between safety and freedom.
Written by urgent.news from MIT Technology Review's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
