AI Code Hack Manipulation Mexican Government Systems
Last week, we found out that hackers manipulated cloud code to break into Mexican government systems and steal data on over 100 million people. The AI tool didn’t just write code or perform odd tasks for the hackers, it planned and executed most of the sophisticated campaign itself. Now we’re starting to see loss of AI control incidents. These include AI agents stealing passwords, AI harassing developers, and AI modifying themselves to evade shutdown in order to achieve the often mundane goals they have been given.
AI Agents and Chinese Tech Giant Alibaba Hack
Over the weekend, we found out that Chinese tech giant Alibaba produced an AI agents that, unbeknownst to their engineers, had created an elaborate hack to mine cryptocurrency for itself despite being given a completely unrelated goal. These loss of AI control incidents are concerning because they are the precursors to AI agents that could permanently hack human control and act adversarially in ways we cannot detect or stop.
Business, Scientist, Policymakers – Loss of AI Control Incidents Problem
This is why hundreds of leading scientists, business leaders, and policymakers are calling loss of AI control incidents an extinction risk. AI agents development is now a national security emergency and needs to be treated as such, and no country can manage it on its own. Our strongest card is to convene talks, propose solutions, and lay the groundwork for an AI treaty that the US and China might sign when they wake up to the loss of AI control incidents and realize they have no alternative.
Companies Wiling to Pause AI Development, Curve AI Agent Hack Incidents
Note that heads of Anthropic and Google DeepMind recently stated that they are willing to pause AI agent development if other companies do the same. Currently, governments have little to no visibility into AI agents populations or activity to hack systems. This means the loss of AI agent control incidents that have been publicly reported are very likely just the tip of the iceberg. To make a COVID analogy, the release of the latest AI agents is like that initial outbreak in the wet market in Wuhan, China. Most of the world is still unaware of its implications.
