AI ‘swarms’ could amplify online misinformation and manipulation, researchers warn

The era of easily detectable botnets is said to be coming to an end. science On Thursday. In the study, researchers warned that misinformation campaigns are moving toward swarms of autonomous AI that can mimic human behavior, adapt in real time and require little to no human supervision, complicating efforts to detect and stop them.

The paper, written by a consortium of researchers including Oxford, Cambridge, UC Berkeley, NYU and the Max Planck Institute, describes a digital environment where tampering is increasingly difficult to identify. Instead of short bursts related to elections or politics, these AI campaigns can maintain a narrative over the long term.

“When governments use these tools, they can suppress dissent or amplify incumbents,” the researchers wrote. “The deployment of defensive AI can therefore only be considered if it is governed by a rigorous, transparent and democratically accountable framework.”

A swarm is a group of autonomous AI agents that work together to solve a problem or complete a goal more efficiently than a single system. Researchers said AI swarms build on existing weaknesses in social media platforms, which often leave users isolated from opposing viewpoints.

“Fake news has been shown to spread faster and more widely than real news, deepening fragmented realities and eroding shared factual baselines,” they wrote. “Recent evidence links participation-optimized curation and polarization. Platform algorithms further degrade the public sphere by amplifying divisive content at the expense of user satisfaction.”

Sean Ren, a computer science professor at the University of Southern California and CEO of Sahara AI, said this shift is already happening on major platforms, with AI-powered accounts becoming increasingly difficult to distinguish from regular users.

“I think more stringent KYC or account identity verification would help a lot here,” Ren said. decryption. “As new accounts become harder to create and spammers become easier to monitor, it becomes much more difficult for agents to use large numbers of accounts for coordinated manipulation.”

Previous influence campaigns relied heavily on scale rather than subtlety, and detection was relatively simple because thousands of accounts were posting identical messages simultaneously. In contrast, research shows that AI swarms demonstrate “unprecedented autonomy, coordination, and scale.”

Ren said content moderation alone would not be enough to stop such a system. The problem, he said, is how platforms manage identities at scale. He said stronger identity verification and restrictions on account creation would make it easier to detect coordinated behavior, even when individual posts appear to be human.

“If agents can only use a small number of accounts to post content, it is much easier to detect suspicious use and ban those accounts,” he said.

No simple fix

The researchers concluded that there is no single solution to this problem, including potential options such as improved detection of statistically anomalous adjustments and greater transparency about automated activity. But he said technical measures alone would not be enough.

According to Ren, even as platforms introduce new technical safeguards, financial incentives remain a persistent driver of coordinated manipulation attacks.

“These swarms of agents are typically controlled by teams or vendors who receive financial incentives from external parties or companies to carry out coordinated manipulations,” he said. “Platforms should implement stronger KYC and spam detection mechanisms to identify and filter accounts that have been manipulated by agents.”