Dot-to-dot: explainable hierarchical reinforcement learning for robotic manipulation
File(s)IROS19_0887_MS.pdf (2.89 MB)
Accepted version
Author(s)
Beyret, Benjamin
Shafti, Seyed Ali
Faisal, Aldo
Type
Conference Paper
Abstract
Robotic systems are ever more capable of automation
and fulfilment of complex tasks, particularly with
reliance on recent advances in intelligent systems, deep learning
and artificial intelligence in general. However, as robots and
humans come closer together in their interactions, the matter
of interpretability, or explainability of robot decision-making
processes for the human grows in importance. A successful
interaction and collaboration would only be possible through
mutual understanding of underlying representations of the
environment and the task at hand. This is currently a challenge
in deep learning systems. We present a hierarchical deep
reinforcement learning system, consisting of a low-level agent
handling the large actions/states space of a robotic system
efficiently, by following the directives of a high-level agent which
is learning the high-level dynamics of the environment and task.
This high-level agent forms a representation of the world and
task at hand that is interpretable for a human operator. The
method, which we call Dot-to-Dot, is tested on a MuJoCo-based
model of the Fetch Robotics Manipulator, as well as a Shadow
Hand, to test its performance. Results show efficient learning
of complex actions/states spaces by the low-level agent, and an
interpretable representation of the task and decision-making
process learned by the high-level agent.
and fulfilment of complex tasks, particularly with
reliance on recent advances in intelligent systems, deep learning
and artificial intelligence in general. However, as robots and
humans come closer together in their interactions, the matter
of interpretability, or explainability of robot decision-making
processes for the human grows in importance. A successful
interaction and collaboration would only be possible through
mutual understanding of underlying representations of the
environment and the task at hand. This is currently a challenge
in deep learning systems. We present a hierarchical deep
reinforcement learning system, consisting of a low-level agent
handling the large actions/states space of a robotic system
efficiently, by following the directives of a high-level agent which
is learning the high-level dynamics of the environment and task.
This high-level agent forms a representation of the world and
task at hand that is interpretable for a human operator. The
method, which we call Dot-to-Dot, is tested on a MuJoCo-based
model of the Fetch Robotics Manipulator, as well as a Shadow
Hand, to test its performance. Results show efficient learning
of complex actions/states spaces by the low-level agent, and an
interpretable representation of the task and decision-making
process learned by the high-level agent.
Date Issued
2020-01-27
Date Acceptance
2019-06-20
Citation
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020, pp.1-6
ISSN
2153-0866
Publisher
IEEE
Start Page
1
End Page
6
Journal / Book Title
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems
Copyright Statement
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Identifier
https://ieeexplore.ieee.org/document/8968488
Source
IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Publication Status
Published
Start Date
2019-11-03
Finish Date
2019-11-08
Coverage Spatial
Macau, China
Date Publish Online
2020-01-27