SINDy-RL for interpretable and efficient model-based reinforcement learning
File(s)
Author(s)
Zolman, Nicholas
Lagemann, Christian
Fasel, Urban
Kutz, J Nathan
Brunton, Steven L
Type
Journal Article
Abstract
Deep reinforcement learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in complex environments, such as stabilizing a tokamak fusion reactor or minimizing the drag force on an object in a fluid flow. However, DRL requires an abundance of training examples and may become prohibitively expensive for many applications. In addition, the reliance on deep neural networks often results in an uninterpretable, black-box policy that may be too computationally expensive to use with certain embedded systems. Recent advances in sparse dictionary learning, such as the sparse identification of nonlinear dynamics (SINDy), have shown promise for creating efficient and interpretable data-driven models in the low-data regime. In this work, we introduce SINDy-RL, a unifying framework for combining SINDy and DRL to create efficient, interpretable, and trustworthy representations of the dynamics model, reward function, and control policy. We demonstrate the effectiveness of our approaches on benchmark control environments and flow control problems, including gust mitigation on a 3D NACA 0012 airfoil at Re = 1000. SINDy-RL achieves comparable performance to modern DRL algorithms using significantly fewer interactions in the environment and results in an interpretable control policy orders of magnitude smaller than a DRL policy.
Date Issued
2025-11-28
Date Acceptance
2025-10-21
Citation
Nature Communications, 2025, 16 (1)
ISSN
2041-1723
Publisher
Nature Portfolio
Journal / Book Title
Nature Communications
Volume
16
Issue
1
Copyright Statement
© The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/41315255
PII: 10.1038/s41467-025-65738-4
Subjects
FLOW
GO
LEVEL
Multidisciplinary Sciences
Science & Technology
Science & Technology - Other Topics
Publication Status
Published
Coverage Spatial
England
Article Number
10714
Date Publish Online
2025-11-28