Multi-Objective Reinforcement Learning for AUV Thruster Failure Recovery
File(s)Ahmadzadeh_SSCI-2014.pdf (1.47 MB)
Accepted version
Author(s)
Ahmadzadeh, Seyed Reza
Kormushev, Petar
Caldwell, Darwin G
Type
Conference Paper
Abstract
This paper investigates learning approaches for discovering fault-tolerant control policies to overcome thruster failures in Autonomous Underwater Vehicles (AUV). The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the vehicle. When a fault is detected and isolated the model of the AUV is reconfigured according to the new condition. To discover a set of optimal solutions a multi-objective reinforcement learning approach is employed which can deal with multiple conflicting objectives. Each optimal solution can be used to generate a trajectory that is able to navigate the AUV towards a specified target while satisfying multiple objectives. The discovered policies are executed on the robot in a closed-loop using AUVs state feedback. Unlike most existing methods which disregard the faulty thruster, our approach can also deal with partially broken thrusters to increase the persistent autonomy of the AUV. In addition, the proposed approach is applicable when the AUV either becomes under-actuated or remains redundant in the presence of a fault. We validate the proposed approach on the model of the Girona500 AUV.
Date Issued
2014-12
Date Acceptance
2014-12-09
Citation
IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014), Proc. IEEE Symposium Series on Computational Intelligence (SSCI 2014), 2014
Publisher
IEEE
Start Page
1
End Page
8
Journal / Book Title
IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014), Proc. IEEE Symposium Series on Computational Intelligence (SSCI 2014)
Copyright Statement
© 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Source
ADPRL 2014
Start Date
2014-12-09
Finish Date
2014-12-12
Coverage Spatial
Orlando, FL