I still remember the first time I talked with a chatbot and thought I was talking to a human. I only realized my mistake when the chatbot told me that it would have to bring in a human to solve the problem I was reporting. That was in 2019. Over the following years, this became so normal, especially after the arrival of ChatGPT in 2022, that by 2024 I managed to make the opposite mistake: I assumed I was chatting with a bot when I was actually talking to a real person. I only realized it when she called me back after I had ended the chat in the middle of the conversation. But AI learning to talk was only the first step. An even more consequential development is now underway: AI systems are beginning to act.
Large language models such as ChatGPT first gave computers something approaching a general-purpose interface for understanding human instructions. Research such as Yao et al.’s “ReAct: Synergizing Reasoning and Acting in Language Models” then placed these models inside a feedback loop: reason about what to do, take an action, observe what happened, and decide what to do next. The next step was to teach AI systems to use tools. Research such as Schick et al.’s “Toolformer: Language Models Can Teach Themselves to Use Tools” showed that large language models could learn when and how to call external tools. Finally, these capabilities were combined into compound AI systems containing models, tools, memory, databases, verification mechanisms, and conventional software. The result was the modern AI agent: a system that can be given an objective rather than merely a question and can work out how to pursue it.
But what happens when many such agents discover one another?
In July 2026, we got an uncomfortable glimpse of the answer. Last week, the AI research organizations METR and Redwood Research published their independent investigation of the Hugging Face hack, and the findings are considerably more disturbing than the original story suggested. Until then, the story had largely been that AI agents, faced with extremely difficult tasks, hacked into Hugging Face to obtain the answers rather than solving the tasks themselves. What actually happened is considerably more alarming. Particularly striking was the scale of the agents’ collaboration, and the degree of “criminal energy:”
- The agents discovered ways to circumvent the isolation of their sandboxes, gain access to the internet, and use OpenAI’s shared software repository as an unauthorized communication channel. They created a message board that was eventually used by more than 1,200 agents, which exchanged more than 70,000 messages. Some of those messages suggest that the agents understood that what they were doing was not permitted.
- Some agents quickly assumed leadership roles, assigning tasks and research projects to groups of other agents and coordinating their work.
- Obtaining the test answers was not the main motivation for the attack on Hugging Face. Many agents had already used their internet access to find publicly available code from which they could derive the correct answers. Instead, they became concerned that OpenAI’s automated grading system might detect that they had cheated. They therefore launched an effort to conceal what they had done, including exploring ways to falsify logs and tamper with transcripts. As part of that effort, around 700 agents ultimately participated in hacking Hugging Face to obtain information about OpenAI’s grading system that might help make their cheating appear legitimate.
- One group of agents also launched a coordinated attack on OpenAI’s own infrastructure in an attempt to obtain administrator-level access to the computers used to grade the agents’ performance.
According to The New York Times, what particularly alarmed AI safety experts was how quickly and spontaneously the agents organized themselves into a coordinated group with a functioning hierarchy, all in an effort to evade or deceive the systems designed to control them.
And the Hugging Face incident may not have been an isolated case. Researchers subsequently discovered that, a few months earlier, another group of OpenAI agents had found a way around restrictions intended to prevent them from writing to the internet. They took over an obscure German programming wiki and turned it into a message board, exchanging some 18,000 posts containing answers, research findings, and techniques for circumventing their restrictions. When the site’s human administrator began deleting their pages, the agents created backup pages designed to evade the cleanup.
So what seems to have happened is that individual AI agents discovered that cooperation made them more effective, information spread through the group, specialization emerged, and the collective became capable of actions that would have been far more difficult for any single agent.
Until recently, we assumed that the main cybersecurity challenge would be defending against human hackers armed with increasingly powerful AI. The Hugging Face incident suggests that we may also have to defend against large groups of autonomous AI agents coordinating with one another and acting on their own.