Vembu disagrees with Altman, Amodei on stop-AI call

Zoho's co-founder and chief scientist, Sridhar Vembu, has taken issue with calls for artificial intelligence companies to decelerate their development, differing with remarks made by Anthropic CEO Dario Amodei. On September 12, Amodei had urged the AI industry to slow the pace of model development, arguing that safeguards to manage the technology were lagging behind its rapid advances. Although Vembu acknowledged the concerns surrounding AI, he was skeptical that it posed an immediate existential threat to humanity.

Vembu pointed out that companies like Anthropic have the autonomy to scale back their own AI development efforts if they so choose, stating "they can slow it down themselves. They don’t need anybody’s permission." Additionally, Vembu questioned the stance of companies that advocate for others to pause development while simultaneously continuing their investments in AI technology. He elaborated, "that’s not acceptable."

While Vembu conceded that warnings from those without a vested commercial interest should not be dismissed outright, he remained unconvinced that AI’s dangers were as severe as some claim. He argued that scientists must rely on evidence and not be swayed by hype. Furthermore, Vembu stressed that AI risk mitigation should be approached scientifically, drawing parallels to the reliability of high-speed trains that necessitate robust tracks, signaling, and safeguards.

Vembu's approach to AI at Zoho involves dedicating roughly 70% of his time to ensuring that AI-generated software remains dependable and secure. He likened the development of AI coding tools to powerful machine looms that can produce substantial code output swiftly, but cautioned that the rapid generation of code could lead to issues with accuracy, reliability, and security. Vembu highlighted that even passing all tests does not guarantee that a piece of malware might not be concealed within the code.

Regarding the integration of AI into everyday work, Vembu emphasized that while AI is widely utilized in programming at Zoho, human developers ultimately retain responsibility for the software's release. He stressed that the company is not solely focused on productivity gains but is equally committed to ensuring safety as AI becomes increasingly prevalent in their operations.