Theory of Mind and the Persuasive Power of AI: Evaluating the Role of AI Chatbots in Reducing Polarization on Topics of US Political Discourse

Abstract

In recent years, the climate for political discourse in the United States has become increasingly heated, with the 2024 presidential election serving as an apparent inflection point. Concurrently, developments in machine learning have enabled large language models (LLMs) to become progressively more human-like, allowing them to take on the role of a social companion. Previous research has suggested that AI chatbots can be successful persuaders in political contexts due to their wealth of factual knowledge and their objective nature. The present study looks to 1) further evaluate AI chatbots as an effective agent in reducing polarized attitudes on political topics, 2) compare their effectiveness to other humans, and 3) to pursue an ideal political identity for the chatbot. College students from Princeton University (N = 42) participated in an 8-round debate on two of their strongest political viewpoints with an AI chatbot (powered by GPT-4o), in which they were informed they were either chatting with the AI or a neighboring human participant through an online message room. Chatbots were prompted to either personify an extreme or moderate opposition to that of the participant. Results from the debates revealed that the chatbot was successful in depolarizing attitudes across all conditions, though differences between conditions were all statistically insignificant. As a follow-up, this paper also includes a textual analysis of the dialogues produced, building on prior research that suggests linguistic styles between humans and AI tend to converge over the course of their conversations.