An unethical optimization principle
File(s)rsos.200462.pdf (728.32 KB)
Published version
Author(s)
Beale, Nicholas
Battey, Heather S
Davison, Anthony C
MacKay, Robert S
Type
Journal Article
Abstract
If an artificial intelligence aims to maximize risk-adjusted return, then under mild conditions it is disproportionately likely to pick an unethical strategy unless the objective function allows sufficiently for this risk. Even if the proportion η of available unethical strategies is small, the probability pU of picking an unethical strategy can become large; indeed, unless returns are fat-tailed pU tends to unity as the strategy space becomes large. We define an unethical odds ratio, Υ (capital upsilon), that allows us to calculate pU from η, and we derive a simple formula for the limit of Υ as the strategy space becomes large. We discuss the estimation of Υ and pU in finite cases and how to deal with infinite strategy spaces. We show how the principle can be used to help detect unethical strategies and to estimate η. Finally we sketch some policy implications of this work.
Date Issued
2020-07-01
Date Acceptance
2020-06-05
Citation
Royal Society Open Science, 2020, 7 (7), pp.1-11
ISSN
2054-5703
Publisher
Royal Society, The
Start Page
1
End Page
11
Journal / Book Title
Royal Society Open Science
Volume
7
Issue
7
Copyright Statement
© 2020 The Authors.
Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.
Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Engineering & Physical Science Research Council (EPSRC)
Identifier
https://royalsocietypublishing.org/doi/10.1098/rsos.200462
Grant Number
EP/P002757/1
EP/P002757/1
Publication Status
Published
Article Number
200462
Date Publish Online
2020-07-01