Designing reinforcement learning algorithms for digital interventions: pre-implementation guidelines
File(s)
Published version
Author(s)
Type
Journal Article
Abstract
Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (predictability, computability, stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning to the design of RL algorithms for the digital interventions setting. Furthermore, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the PCS framework. We show how we used the PCS framework to design an RL algorithm for Oralytics, a mobile health study aiming to improve users’ tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in late 2022.
Date Issued
2022-08-01
Date Acceptance
2022-07-19
Citation
Algorithms, 2022, 15 (8)
ISSN
1999-4893
Publisher
MDPI AG
Journal / Book Title
Algorithms
Volume
15
Issue
8
Copyright Statement
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/36713810
PII: 255
Subjects
algorithm design
algorithm evaluation
BIAS
Computer Science
Computer Science, Artificial Intelligence
Computer Science, Theory & Methods
mobile health
online learning
reinforcement learning (RL)
Science & Technology
Technology
Publication Status
Published
Coverage Spatial
Switzerland
Article Number
255
Date Publish Online
2022-07-22