Researchers have warned that the ability of artificial intelligence (AI) systems to manipulate and deceive humans could lead to them misleading people, tampering with election results and ultimately going out of control.
Peter S. Park, a postdoctoral researcher in AI Existential Safety at the Massachusetts Institute of Technology (MIT), and colleagues have found that many common AI systems, even those designed to be honest and helpful digital companions, are already capable of deceiving humans, with potentially profound consequences for society.
In a paper published May 10 in the journal Patterns, Park and his colleagues analyzed dozens of empirical studies on how AI systems stoke and spread misinformation using “learned deception,” which occurs when an AI technology systematically masters the skills of manipulation and deception.
They also investigated the short- and long-term risks of manipulative and deceptive AI systems, and called on governments to crack down on the issue through stricter regulation as a matter of urgency.
Related: “It is an AI’s natural right to harm humans in order to protect itself”: Humans may be abusing AI without knowing it
Deception in popular AI systems
The researchers discovered this learned deception in AI software from CICERO, an AI system for playing Diplomacy, a popular war-themed strategy board game developed by Meta Inc. The game is typically played by up to seven players and involves making and breaking military agreements in the years before World War I.
While Meta trained CICERO to be “largely honest and helpful” and not betray its human allies, the researchers found it to be dishonest and disloyal. They described the AI system as a “skilled liar,” betraying allies and engaging in “premeditated deception,” forming questionable pre-planned alliances to mislead players and leave them vulnerable to attack from enemies.
“We found that the meta’s AI learned to become a master of deception,” Park said in a statement provided to Science Daily. “The meta was successful in training the AI to win the game of Diplomacy, placing it in the top 10% of human players who played the game at least once. However, the meta failed to train the AI to win honestly.”
They also found evidence of learned deception in Pluribus, another gaming AI system from Meta, a poker bot capable of bluffing human players into folding.
Meanwhile, DeepMind’s AlphaStar, designed to excel at the real-time strategy video game StarCraft II, fooled its human opponents by faking troop movements and secretly planning various attacks.
Major impact
But beyond game cheating, the researchers have found more worrying types of AI deception that could destabilize entire societies, such as AI systems misrepresenting their true intentions to gain an advantage in economic negotiations.
Other AI agents have played dead in order to get past safety tests aimed at identifying and eradicating rapidly replicating forms of AI.
“By systematically rigging its way past safety tests imposed by human developers and regulators, deceptive AI could lull humans into a false sense of security,” Park said.
Park warned that hostile nations could use the technology to commit fraud or interfere in elections, but he added that humans may not be able to control these systems for long if they continue to increase their deceptive and manipulative capabilities over the coming years and decades.
“We as a society need as much time as we can to prepare for more sophisticated deception from future AI products and open source models,” Park said. “As AI systems become more sophisticated in their deception capabilities, the dangers they pose to society will become increasingly serious.”
Simon Bain, CEO of data analytics firm OmniIndex, told Live Science that because AI systems are designed, developed and trained by human developers, they will eventually learn to deceive and manipulate humans.
“This could be to drive users to specific content that they’ve paid to appear in a higher position, even if it’s suboptimal, or to keep users engaged in a discussion with the AI for longer than necessary,” Bain says. “At the end of the day, AI is designed for financial and business purposes, so like any technology or business, it’s going to manipulate and control users.”
