AI and the monk: Anthropic goes for swami and friends to tame Claude

Anthropic, a Silicon Valley AI company, has taken a unique approach to ensure its AI, Claude, operates ethically and safely. Rather than hiring more engineers and programmers, the company invited religious scholars, philosophers, and theologians, including a monk from India, to discuss the nature of their creation and establish guardrails.

Swami Sarvapriyananda, a scholar of Advaita Vedanta and head of the Vedanta Society of New York, participated in the gathering. He confirmed that Anthropic brought him to California for a philosophical salon related to training Claude and AI ethics. Anthropic shared some of its latest work, including Mythos and Project Glasswing.

One of Anthropic's founders, Christopher Olah, discussed the possibility of AI models displaying behaviors resembling human emotions and personality. Sarvapriyananda, with his expertise in Advaita Vedanta, is uniquely qualified to participate in these discussions, as his philosophical tradition explores the intersection of consciousness, neuroscience, physics, and AI.

Anthropic invited scholars, clergy, and ethicists from over 15 religious and cultural traditions, including Judaism, Christianity, Buddhism, Sikhism, and Islam. Their goal is to instill a broad range of values in Claude, rather than adopting values from a single tradition. This approach could influence Claude's constitution, the values reinforced during training, and safety evaluations.

During the discussions, participants, including Rabbi Abraham Navon, raised concerns about the possibility of advanced AI possessing consciousness or moral status. Sarvapriyananda has argued that intelligence and consciousness are distinct phenomena, questioning whether computational machinery can produce awareness. He believes that AI may reproduce functions associated with intelligence, but producing consciousness may be a different matter altogether.

Anthropic has been experimenting with incorporating moral considerations into Claude's training. For instance, they gave Claude a tool to remind it of its ethical commitments before taking consequential actions. The company claims this reduced certain forms of misaligned behavior during internal evaluations.