Abstract
This study investigates the use of LLMs, specifically ChatGPT-4o, to enhance the moderation of online deliberative processes. Traditionally, decision-making has been controlled by small groups, often excluding the vital insights that crowd intelligence can provide. As global challenges grow more complex, broader and more inclusive participation is essential. While online platforms allow for such large-scale participation, they also face significant issues, including content fragmentation, low signal-to-noise ratios, and inefficient argumentation. Human moderators can address these challenges, but scaling them is prohibitively costly. This research introduces a more scalable solution by leveraging LLMs to automate critical moderation tasks, including unbundling multiple ideas, categorizing them into solutions, metrics, and barriers, and implementing efficient argument mining and classification techniques. Additionally, it evaluates the effectiveness of different prompting styles in optimizing moderation. The findings demonstrate that LLMs can successfully moderate key aspects of large-scale online deliberations, such as unbundling and categorization, improving the structure of discussions and representing a significant step forward in collective decision-making.
Using LLMs to automate moderation tasks:
- Unbundling multiple ideas
- Extracting distinct ideas from a single user post, and categorizing them into solutions, metrics, and barriers
- Few shot learning did well
- Implementing efficient argument mining
- Identifying and extracting all distinct arguments related to ideas/answers discussed
- Few shot learning did well, but did as well as zero-shot
- Argument classification
- Classifying pro or con
- Weakest task