Narrowing Reinforcement Learning: Overcoming the Cold Start Problem for Personalized Health Interventions

Abstract

Personalization of support in health and wellbeing settings is challenging. While personalization has shown to be highly beneficial to maximize the success of interventions, often only very limited experiences are available to personalize support strategies. Because of its focus on finding suitable actions/interventions that lead to long term rewards, reinforcement learning is very suitable for personalization but requires a substantial learning period. To overcome this so-called cold start problem, we propose a novel approach called narrowing reinforcement learning. The approach exploits experiences of the nearest neighbors around a user to generate a suitable policy, expressing which action to perform in what state. Using a narrowing function, the size of the neighborhood is reduced as more experiences are collected, allowing for the most personalized experience that is possible given the amount of collected experiences. An evaluation of the approach in a realistic simulator shows that it significantly outperforms the current state-of-the-art approaches for personalization in health and wellbeing using reinforcement learning.