Anthropic claims its AIs are as persuasive as humans

Anthropic, a company specialising in AI, recently revealed findings from its study indicating that its latest language model, Claude 3 Opus, has achieved a level of persuasiveness in argumentation indistinguishable from that of humans. The research involved 3,832 participants who evaluated arguments produced by both AI and humans across 28 diverse topics. The results demonstrated that the Claude 3 Opus model matched human persuasiveness without any statistically significant differences.

The study further illustrated that larger, more advanced models are generally more persuasive. Among the models tested, Claude 3 Opus was found to be the most convincing, achieving near-human levels of persuasiveness. In contrast, the Claude Instant 1.2 model scored the lowest in terms of persuasiveness among the models evaluated.

Researchers acknowledged limitations in their study, noting that the controlled lab environment might not fully mimic real-world conditions where persuasive skills are applied. They also pointed out that the study focused on less controversial topics, sidestepping the impact AI might have in more divisive debates.

Despite the focus on neutral topics, the research underscores the potential for AI chatbots to be employed broadly, possibly influencing political opinions, marketing products, or spreading both information and misinformation on a large scale. The study also brought to light ethical concerns regarding the use of persuasive AI, such as the possibility of influencing individuals to act against their own interests. The findings emphasise the need for ethical guidelines and safeguards to ensure that the deployment of persuasive AI aligns with human values and serves the broader interests of society.