SurveyPilot: an Agentic Framework for Automated Human Opinion Collection from Social Media

Opinion survey research is a crucial method used by social scientists for understanding societal beliefs and behaviors. Traditional methodologies often entail high costs and limited scalability, while current automated methods such as opinion synthesis exhibit severe biases and lack traceability. In this paper, we introduce S UR - VEY P ILOT , a novel finite-state orchestrated agentic framework that automates the collection and analysis of human opinions from social media platforms. S URVEY P ILOT addresses the limitations of pioneering approaches by (i) providing transparency and traceability in each state of opinion collection and (ii) incorporating several techniques for mitigating biases, notably with a novel genetic algorithm for improving result diversity. Our extensive experiments reveal that S URVEY P ILOT achieves a close alignment with authentic survey re-sults across multiple domains, observing average relative improvements of 68.98% and 51.37% when comparing to opinion synthesis and agent-based approaches. Implementation of S URVEY P ILOT is available on https: //github.com/thanhpv2102/SurveyPilot