Reinforcement learning with adaptive regularization for safe control of critical systems
Author(s)
Tian, Haozhe
Hamedmoghadam, Homayoun
Shorten, Robert
Ferraro, Pietro
Type
Conference Paper
Abstract
Reinforcement Learning (RL) is a powerful method for controlling dynamic systems, but its learning mechanism can lead to unpredictable actions that undermine the safety of critical systems. Here, we propose RL with Adaptive Regularization (RL-AR), an algorithm that enables safe RL exploration by combining the RL policy with a policy regularizer that hard-codes the safety constraints. RL-AR performs policy combination via a “focus module,” which determines the appropriate combination depending on the state-relying more on the safe policy regularizer for less-exploited states while allowing unbiased convergence for well-exploited states. In a series of critical control applications, we demonstrate that RL-AR not only ensures safety during training but also achieves a return competitive with the standards of model-free RL that disregards safety.
Date Issued
2024
Date Acceptance
2024-12-01
Citation
Advances in Neural Information Processing Systems, 2024, 37, pp.2528-2557
ISBN
9798331314385
ISSN
1049-5258
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Start Page
2528
End Page
2557
Journal / Book Title
Advances in Neural Information Processing Systems
Volume
37
Copyright Statement
© 2025 Neural Information Processing Systems Foundation, Inc. (NeurIPS).
Source
NeurIPS 2024
Publication Status
Published
Start Date
2024-12-10
Finish Date
2024-12-15
Coverage Spatial
Vancouver, Canada