Sim-to-real reinforcement learning for deformable object manipulation
File(s)
Accepted version
Author(s)
Matas, Jan
James, Stephen
Davison, Andrew
Type
Conference Paper
Abstract
We have seen much recent progress in rigid object manipulation, but in-
teraction with deformable objects has notably lagged behind. Due to the large con-
figuration space of deformable objects, solutions using traditional modelling ap-
proaches require significant engineering work. Perhaps then, bypassing the need
for explicit modelling and instead learning the control in an end-to-end manner
serves as a better approach? Despite the growing interest in the use of end-to-end
robot learning approaches, only a small amount of work has focused on their ap-
plicability to deformable object manipulation. Moreover, due to the large amount
of data needed to learn these end-to-end solutions, an emerging trend is to learn
control policies in simulation and then transfer them over to the real world. To-
date, no work has explored whether it is possible to learn and transfer deformable
object policies. We believe that if sim-to-real methods are to be employed fur-
ther, then it should be possible to learn to interact with a wide variety of objects,
and not only rigid objects. In this work, we use a combination of state-of-the-art
deep reinforcement learning algorithms to solve the problem of manipulating de-
formable objects (specifically cloth). We evaluate our approach on three tasks —
folding a towel up to a mark, folding a face towel diagonally, and draping a piece
of cloth over a hanger. Our agents are fully trained in simulation with domain
randomisation, and then successfully deployed in the real world without having
seen any real deformable objects.
teraction with deformable objects has notably lagged behind. Due to the large con-
figuration space of deformable objects, solutions using traditional modelling ap-
proaches require significant engineering work. Perhaps then, bypassing the need
for explicit modelling and instead learning the control in an end-to-end manner
serves as a better approach? Despite the growing interest in the use of end-to-end
robot learning approaches, only a small amount of work has focused on their ap-
plicability to deformable object manipulation. Moreover, due to the large amount
of data needed to learn these end-to-end solutions, an emerging trend is to learn
control policies in simulation and then transfer them over to the real world. To-
date, no work has explored whether it is possible to learn and transfer deformable
object policies. We believe that if sim-to-real methods are to be employed fur-
ther, then it should be possible to learn to interact with a wide variety of objects,
and not only rigid objects. In this work, we use a combination of state-of-the-art
deep reinforcement learning algorithms to solve the problem of manipulating de-
formable objects (specifically cloth). We evaluate our approach on three tasks —
folding a towel up to a mark, folding a face towel diagonally, and draping a piece
of cloth over a hanger. Our agents are fully trained in simulation with domain
randomisation, and then successfully deployed in the real world without having
seen any real deformable objects.
Date Issued
2018-10-29
Date Acceptance
2018-09-02
Citation
Proceedings of Machine Learning Research, 2018, 87, pp.734-743
Publisher
PMLR
Start Page
734
End Page
743
Journal / Book Title
Proceedings of Machine Learning Research
Volume
87
Copyright Statement
© 2018 by the author(s). Available under a CC-BY Attribution Licence (http://creativecommons.org/licenses/by/4.0)
Source
Conference on Robot Learning 2018
Publication Status
Published
Start Date
2018-10-29
Finish Date
2018-10-31
Coverage Spatial
Zurich, Switzerland