Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective
Author(s)
Story, G
Vlaev, I
Seymour, B
Darzi, A
Dolan, R
Type
Journal Article
Abstract
The tendency to make unhealthy choices is hypothesized to be related to an individual’s temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a “model-based” (or goal-directed) system and a “model-free” (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes.
Date Issued
2014-03-12
Online Publication Date
2014-03-12
2017-03-10T11:15:53Z
Date Acceptance
2014-02-21
ISSN
1662-5153
Publisher
Frontiers Media
Journal / Book Title
Frontiers in Behavioral Neuroscience
Volume
8
Copyright Statement
© 2014 Story, Vlaev, Seymour, Darzi and Dolan. This is an open-
access article distributed under the terms of the Creative Commons Attribution
License (https://creativecommons.org/licenses/by/3.0/). The use, distribution or reproduction in other forums is permit-
ted, provided the original author(s) or licensor are credited and that the original
publication in this journal is cited, in accordance with accepted academic prac-
tice. No use, distribution or reproduction is permitted which does not comply with
these terms.
access article distributed under the terms of the Creative Commons Attribution
License (https://creativecommons.org/licenses/by/3.0/). The use, distribution or reproduction in other forums is permit-
ted, provided the original author(s) or licensor are credited and that the original
publication in this journal is cited, in accordance with accepted academic prac-
tice. No use, distribution or reproduction is permitted which does not comply with
these terms.
Source Database
web-of-science
Sponsor
National Institute for Health Research
Grant Number
NF-SI-0510-10186
Subjects
Science & Technology
Life Sciences & Biomedicine
Behavioral Sciences
Neurosciences
Neurosciences & Neurology
BEHAVIORAL SCIENCES
NEUROSCIENCES
discounting
health
addiction
model-based
model-free
habit
preference reversal
hyperbolic
CIGARETTE-SMOKING STATUS
COCAINE-DEPENDENT OUTPATIENTS
INTERTEMPORAL DECISION-MAKING
INDIVIDUAL TIME-PREFERENCES
COST-EFFECTIVENESS ANALYSIS
PREVENTIVE HEALTH BEHAVIOR
BODY-MASS INDEX
DELAYED REWARDS
SELF-CONTROL
MODEL-FREE
Publication Status
Published
Article Number
76