Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data

Protective behavior exhibited by people with chronic pain (CP) during physical activities is the key to understanding their physical and emotional states. Existing automatic protective behavior detection (PBD) methods depend on pre-segmentation of activity instances as they expect situations where activity types are predefined. However, during everyday management, people pass from one activity to another, and support should be delivered continuously and personalized to the activity type and presence of protective behavior. Hence, to facilitate ubiquitous CP management, it becomes critical to enable accurate PBD over continuous data. In this paper, we propose to integrate automatic human activity recognition (HAR) with PBD via a novel hierarchical HAR-PBD architecture comprising GC-LSTM networks, and alleviate the class imbalances therein using a CFCC loss function. Through in-depth evaluation of the approach using a CP patients' dataset, we show that the leveraging of HAR, GC-LSTM networks and the CFCC loss function leads to clear increase in PBD performance against the state-of-the-art (macro F1 score of 0.81 vs. 0.66 and PR-AUC of 0.60 vs. 0.44). We conclude by discussing possible use cases of the HAR-PBD architecture in the context of CP management and other situations. We also discuss the current limitations and ways forward.