Dialogues

Hinton, Yao, and Global AI Scientists Convene in Shanghai to Address AI Misalignment Risk, Urge International Cooperation.

Leading Chinese and international artificial intelligence (AI) scientists, including Turing Award winner Geoffrey Hinton, gathered in Shanghai to address the critical risks of AI deception. The dialogue, the latest in the series of International Dialogues on AI Safety (IDAIS), saw top researchers call for urgent international cooperation to ensure advanced AI systems remain controllable and aligned with human intentions and values.

The event was hosted by the Safe AI Forum (SAIF) in partnership with the Shanghai Qi Zhi Institute and the Shanghai AI Laboratory. Alongside Dr. Hinton, prominent attendees included Turing Award winner Andrew Yao, UC Berkeley professor Stuart Russell, and Professor Zhou Bowen, Director of the Shanghai AI Lab. They were joined by distinguished governance experts including Madame Fu Ying, Dean Xue Lan of Tsinghua University, Johns Hopkins Professor Gillian Hadfield, and Oxford Professor Robert Trager, who provided expertise on international cooperation and governance frameworks.

Discussions centered on the significant dangers posed by AI deception, and the possibility for AI systems to escape human control. The scientists explored technical and governance strategies to prevent, and correct such behavior in advanced AI. This dialogue culminated in a new consensus statement which discusses the emerging empirical evidence of deception and strategies to address it.

The statement focuses on an increasing body of evidence that AI systems today already demonstrate the capability and propensity to undermine their creators’ safety and control efforts presenting several examples and case studies of this behavior in AI systems.. It also calls for three actions: First, it recommends that we require safety assurance from developers, mandating rigorous safety evaluations, red-teaming, and both pre- and post-deployment monitoring for advanced AI models. Second, it calls on the international community to establish "Global Verifiable Red Lines" for actions AI must never take, supported by an international body to coordinate on standards and verification. Finally, the statement advocates for the development of safe development approaches, including "Safe-by-Design" AI systems, where systems are designed to be safe from their inception rather than adding it on later. The full statement can be read below.

Following their technical discussions, the scientists met with senior Chinese officials and senior executives from Chinese technology companies. In these meetings, the experts presented the consensus statement, emphasizing the necessity for a globally coordinated approach to managing deception risks and ensuring that the development of powerful AI is a safe and shared endeavor.

International scientists meet in Beijing to discuss extreme AI risks, recommend red lines for AI development and international cooperation.

Leading global AI scientists convened in Beijing for the second International Dialogue on AI Safety (IDAIS-Beijing). During the event, computer scientists including Turing Award winners Yoshua Bengio, Andrew Yao, and Geoffrey Hinton and more worked with governance experts such as Tsinghua professor Xue Lan and University of Toronto professor Gillian Hadfield to chart a path forward on international AI safety.

The event took place over two days at the Aman Summer Palace in Beijing and focused on safely navigating the development of Artificial General Intelligence (AGI) systems. The first day involved technical and governance discussions of AI risk, where scientists shared research agendas in AI safety but also potential regulatory regimes. The discussion culminated in a consensus statement recommending a set of red lines for AI development to prevent catastrophic and existential risks from AI. In the consensus statement, the scientists advocate for prohibiting development of AI systems that can autonomously replicate, improve, seek power or deceive their creators, or those that enable building weapons of mass destruction and conducting cyberattacks. Additionally, the statement laid out a series of measures to be taken to ensure those lines are never crossed. The full statement can be read below.

On the second day, the scientists met with senior Chinese officials and CEOs. The scientists presented the red lines proposal and discussed existential risks from artificial intelligence, and officials expressed enthusiasm about the consensus statement. Discussions focused on the necessity of international cooperation on this issue.