Söderström, Arne, Shatte, Adrian, and Fuller-Tyszkiewicz, Matthew (2021) Can intelligent agents improve data quality in online questiosnnaires? A pilot study. Behavior Research Methods, 53. pp. 2238-2251.

PDF (Published Version) - Published Version Restricted to Repository staff only

Abstract

We explored the utility of chatbots for improving data quality arising from collection via sonline surveys. Three-hundred Australian adults sampled via Prolific Academic were randomized across chatbot-supported or unassisted online questionnaire conditions. The questionnaire comprised validated measures, along with challenge items formulated to be confusing yet aligned with the validated targets. The chatbot condition provided optional assistance with item clarity via a virtual support agent. Chatbot use and user satisfaction were measured through session logs and user feedback. Data quality was operationalized as between-group differences in relationships among validated and challenge measures. Findings broadly supported chatbot utility for online surveys, showing that most participants with chatbot access utilized it, found it helpful, and demonstrated modestly improved data quality (vs. controls). Absence of confusion for one challenge item is believed to have contributed to an underestimated effect. Findings show that assistive chatbots can enhance data quality, will be utilized by many participants if available, and are perceived as beneficial by most users. Scope constraints for this pilot study are believed to have led to underestimated effects. Future testing with longer-form questionnaires incorporating expanded item difficulty may further understanding of chatbot utility for survey completion and data quality.

Item ID:	81639
Item Type:	Article (Research - C1)
ISSN:	1554-3528
Keywords:	chatbot; autonomous conversational agent; online survey research; questionnaire item confusion; response accuracy; data quality
Copyright Information:	© The Psychonomic Society, Inc. 2021
Date Deposited:	06 Feb 2024 03:15
FoR Codes:	46 INFORMATION AND COMPUTING SCIENCES > 4601 Applied computing > 460199 Applied computing not elsewhere classified @ 40% 46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460202 Autonomous agents and multiagent systems @ 60%
SEO Codes:	22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220499 Information systems, technologies and services not elsewhere classified @ 100%
Downloads:	Total: 1
	More Statistics