Gaussian Processes for Data-Efficient Learning in Robotics and Control
File(s)
Accepted version
Author(s)
Deisenroth, MP
Fox, D
Rasmussen, CE
Type
Journal Article
Abstract
Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven
learning allows to reduce the amount of engineering knowledge, which is otherwise required. However, autonomous reinforcement
learning (RL) approaches typically require many interactions with the system to learn controllers, which is a practical limitation in real
systems, such as robots, where many interactions can be impractical and time consuming. To address this problem, current learning
approaches typically require task-specific knowledge in form of expert demonstrations, realistic simulators, pre-shaped policies, or
specific knowledge about the underlying dynamics. In this article, we follow a different approach and speed up learning by extracting
more information from data. In particular, we learn a probabilistic, non-parametric Gaussian process transition model of the system.
By explicitly incorporating model uncertainty into long-term planning and controller learning our approach reduces the effects of model
errors, a key problem in model-based learning. Compared to state-of-the art RL our model-based policy search method achieves an
unprecedented speed of learning. We demonstrate its applicability to autonomous learning in real robot and control tasks.
learning allows to reduce the amount of engineering knowledge, which is otherwise required. However, autonomous reinforcement
learning (RL) approaches typically require many interactions with the system to learn controllers, which is a practical limitation in real
systems, such as robots, where many interactions can be impractical and time consuming. To address this problem, current learning
approaches typically require task-specific knowledge in form of expert demonstrations, realistic simulators, pre-shaped policies, or
specific knowledge about the underlying dynamics. In this article, we follow a different approach and speed up learning by extracting
more information from data. In particular, we learn a probabilistic, non-parametric Gaussian process transition model of the system.
By explicitly incorporating model uncertainty into long-term planning and controller learning our approach reduces the effects of model
errors, a key problem in model-based learning. Compared to state-of-the art RL our model-based policy search method achieves an
unprecedented speed of learning. We demonstrate its applicability to autonomous learning in real robot and control tasks.
Date Issued
2014-02-01
Citation
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
ISSN
0162-8828
Publisher
IEEE
Start Page
408
End Page
423
Journal / Book Title
IEEE Transactions on Pattern Analysis and Machine Intelligence
Volume
37
Issue
2
Copyright Statement
© 2013 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution
requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Publication Status
Accepted