Evaluation of In-Person Counseling Strategies To Develop Physical Activity Chatbot for Women

Artificial intelligence chatbots are the vanguard in technology-based intervention to change people's behavior. To develop intervention chatbots, the first step is to understand natural language conversation strategies in human conversation. This work introduces an intervention conversation dataset collected from a real-world physical activity intervention program for women. We designed comprehensive annotation schemes in four dimensions (domain, strategy, social exchange, and task-focused exchange) and annotated a subset of dialogs. We built a strategy classifier with context information to detect strategies from both trainers and participants based on the annotation. To understand how human intervention induces effective behavior changes, we analyzed the relationships between the intervention strategies and the participants' changes in the barrier and social support for physical activity. We also analyzed how participant's baseline weight correlates to the amount of occurrence of the corresponding strategy. This work lays the foundation for developing a personalized physical activity intervention bot. The dataset and code are available at https://github.com/KaihuiLiang/physical-activity-counseling

Discovering Chatbot's Self-Disclosure's Impact on User Trust, Affinity, and Recommendation Effectiveness

In recent years, chatbots have been empowered to engage in social conversations with humans and have the potential to elicit people to disclose their personal experiences, opinions, and emotions. However, how and to what extent people respond to chabots' self-disclosure remain less known. In this work, we designed a social chatbot with three self-disclosure levels that conducted small talks and provided relevant recommendations to people. 372 MTurk participants were randomized to one of the four groups with different self-disclosure levels to converse with the chatbot on two topics, movies, and COVID-19. We found that people's self-disclosure level was strongly reciprocal to a chatbot's self-disclosure level. Chatbots' self-disclosure also positively impacted engagement and users' perception of the bot and led to a more effective recommendation such that participants enjoyed and agreed more with the recommendations.

HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversations

Open-domain dialog systems have a user-centric goal: to provide humans with an engaging conversation experience. User engagement is one of the most important metrics for evaluating open-domain dialog systems, and could also be used as real-time feedback to benefit dialog policy learning. Existing work on detecting user disengagement typically requires hand-labeling many dialog samples. We propose HERALD, an efficient annotation framework that reframes the training data annotation process as a denoising problem. Specifically, instead of manually labeling training samples, we first use a set of labeling heuristics to label training samples automatically. We then denoise the weakly labeled data using the Shapley algorithm. Finally, we use the denoised data to train a user engagement detector. Our experiments show that HERALD improves annotation efficiency significantly and achieves 86% user disengagement detection accuracy in two dialog corpora.