US-China AI talks must ensure safety of both tech and business

In recent US-China talks on AI, both nations have agreed to establish a formal dialogue and communication channel. As the next AI meeting approaches in November, cooperation between the two countries is crucial. The AI sector is rapidly evolving, with American tech firms investing heavily in chips, data centers, and energy. This investment, while significant, could lead to financial instability if the AI market crashes.

Currently, only a minority of Americans see more benefits than drawbacks in AI products, compared to a larger percentage in China. Concerns about rogue AI agents and cyberattacks have grown, highlighting the need for safety measures. Leading AI developers and scientists are calling for safeguards, such as independent evaluations and third-party testing.

The upcoming US-China AI dialogue should focus on both managing risks and building a sustainable business case for AI. China's strengths in testing, refining, and commercializing AI applications across various sectors could complement American developers' need for revenue-generating applications. A balanced approach, incorporating both American frontier models and efficient Chinese open-weight models, could provide cost-effective solutions for routine tasks and high-priority problems.

Addressing AI risks, such as the development of AI-enabled autonomous weapons, requires a commitment to reciprocal no-first-use. Finally, a shared UN-linked governance framework and an annual US-China AI summit could facilitate collaboration and ensure that the technology's moral value aligns with human values.