1182
OpenAI conducted a large-scale study to assess the persuasiveness of artificial intelligence arguments by comparing the model's responses o3-mini with comments from subreddit users r/ChangeMyView, which specializes in debates and changing perspectives. The results showed that the AI was able to outperform humans 82% of the time, although it did not yet reach the level of “superhuman” persuasiveness, which OpenAI defines as the 95th percentile.
Forum r/ChangeMyView, which has 3,8 million members, creates a unique database of arguments as users mark responses that really changed their perspective with a special “delta” icon. OpenAI used these comments as a control sample, comparing them to responses o3-mini using expert evaluation on a five-point scale.
Previous versions of ChatGPT were significantly less convincing to humans: for example, GPT-3.5 reached only the 2022th percentile in 38. However, gradual improvements in models have led to an increase in this indicator: o1-mini in September 2023 reached 77%, and o1 – almost 80%. In turn, o3-mini exceeded the average human persuasiveness in 82% of tests, indicating significant progress in the development of language models.
It’s important to note that the evaluation was based on random responses from Reddit users, and even if the AI received a rating just slightly higher than a human, it was counted as a success. However, the experiment did not show how much the AI’s arguments actually changed readers’ minds.
OpenAI believes o3-mini "medium risk" model, as its persuasiveness is roughly equivalent to that of an average person, which can influence media, election campaigns and information manipulation. However, the company warns that in the future, extremely persuasive models could pose a danger to democracy, intelligence and government, as they can coerce people to act contrary to their beliefs.
To prevent misuse of the technology, OpenAI is already developing controls, including monitoring the use of AI in political and information campaigns. The company also highlights the risks of creating false public consensus on a large scale (astroturfing) through automatic generation of persuasive texts. Although o3-mini has not yet reached a critical level of influence, progress in the development of language models requires increased attention to the ethical and social consequences of their use.
