Abstract: Psychological stress is a major contributor to the adoption of unhealthy behaviors, which in turn accounts for 41% of global cardiovascular disease burden. While the proliferation of mobile health apps has offered promise to stress management, these apps do not provide micro-level feedback with regard to how to adjust one’s behaviors to achieve a desired health outcome. In this paper, we formulate the task of multi-stage stress management as a sequential decision-making problem and explore the application of reinforcement learning to provide micro-level feedback for stress reduction. Specifically, we incorporate a multi-stage threshold selection into Q-learning to derive an interpretable form of a recommendation policy for behavioral coaching. We apply this method on an observational dataset that contains Fitbit ActiGraph measurements and self-reported stress levels. The estimated policy is then used to understand how exercise patterns may affect users’ psychological stress levels and to perform coaching more effectively.

Learning Objective 1: To construct interpretable and adaptive decision rules for multi-stage stress management using the Multi-stage Threshold Q-learning method.

Learning Objective 2: To study the effect of different numbers of intervention stages on stress reduction.


Xinyu Hu (Presenter)
Columbia University

Pei-Yun Hsueh, IBM T.J. Watson Research Center
Ching-Hua Chen, IBM T.J. Watson Research Center
Keith Diaz, Columbia University
Ying-Kuen Cheung, Columbia University
Min Qian, Columbia University

Presentation Materials: