Overcoming error-in-variable or stiffness problem in SINDy-like data-driven model discovery
File(s)
Accepted version
Author(s)
Fung, Lloyd
Type
Journal Article
Abstract
Despite the recent proliferation of machine learning methods like SINDy that promise automatic discovery of
governing equations from time-series data, there remain significant challenges to discovering models from
noisy datasets. One reason is that the linear regression underlying these methods assumes that all noise resides in the training target (the regressand), which is the time derivative, whereas the measurement noise is in the states (the regressors). Recent methods like modified-SINDy and DySMHO address this error-in-variable problem by leveraging information from the model’s temporal evolution, but they are also imposing the equation as a hard constraint, which effectively assumes no error in the regressand. Without relaxation,
this hard constraint prevents assimilation of data longer than Lyapunov time. Instead, the fulfilment of the
model equation should be treated as a soft constraint to account for the small yet critical error introduced by
numerical truncation. The uncertainties in both the regressor and the regressand invite the use of orthogonal distance regression (ODR). By incorporating ODR with the Bayesian framework for model selection, we introduce a novel method for model discovery, termed ODR-BINDy, and assess its performance against current SINDy variants using the Lorenz63, Rössler, and Van Der Pol systems as case studies. Our findings indicate that ODR-BINDy consistently outperforms all existing methods in recovering the correct model from sparse and noisy datasets. For instance, our ODR-BINDy method reliably recovers the Lorenz63 equation from data with noise contamination levels of up to 30%.
governing equations from time-series data, there remain significant challenges to discovering models from
noisy datasets. One reason is that the linear regression underlying these methods assumes that all noise resides in the training target (the regressand), which is the time derivative, whereas the measurement noise is in the states (the regressors). Recent methods like modified-SINDy and DySMHO address this error-in-variable problem by leveraging information from the model’s temporal evolution, but they are also imposing the equation as a hard constraint, which effectively assumes no error in the regressand. Without relaxation,
this hard constraint prevents assimilation of data longer than Lyapunov time. Instead, the fulfilment of the
model equation should be treated as a soft constraint to account for the small yet critical error introduced by
numerical truncation. The uncertainties in both the regressor and the regressand invite the use of orthogonal distance regression (ODR). By incorporating ODR with the Bayesian framework for model selection, we introduce a novel method for model discovery, termed ODR-BINDy, and assess its performance against current SINDy variants using the Lorenz63, Rössler, and Van Der Pol systems as case studies. Our findings indicate that ODR-BINDy consistently outperforms all existing methods in recovering the correct model from sparse and noisy datasets. For instance, our ODR-BINDy method reliably recovers the Lorenz63 equation from data with noise contamination levels of up to 30%.
Date Acceptance
2026-07-10
Citation
ACM Transactions on AI in Science
Journal / Book Title
ACM Transactions on AI in Science
Copyright Statement
Copyright This paper is embargoed until publication. Once published the author’s accepted manuscript will be made available under a CC-BY License in accordance with Imperial’s Research Publications Open Access policy (www.imperial.ac.uk/oa-policy).
License URL
Publication Status
Accepted