SplitWise regression for capturing nonlinear effects in interpretable model selection
File(s)
Published version
Author(s)
Kurbucz, Marcell T
Tzivanakis, Nikolaos
Sari Aslam, Nilufer
Sykulski, Adam M
Type
Journal Article
Abstract
Capturing nonlinear relationships while maintaining interpretability remains a persistent challenge
in regression modeling. We introduce SplitWise, a stepwise regression framework that adaptively transforms numeric predictors into threshold-based binary features using shallow decision trees—only when such transformations improve model fit according to the Akaike or Bayesian Information Criterion. This design preserves the transparency of linear models while flexibly capturing threshold-based nonlinear effects, positioning SplitWise between classical linear and interpretable nonlinear
regression. SplitWise retains a single, globally linear equation that selectively incorporates data-driven thresholds—yielding models that remain straightforward to interpret and verify. Across synthetic scenarios with nonlinear signal patterns, SplitWise reduced median RMSE by 7–14% relative to the best-performing interpretable linear baseline and improved variable-selection accuracy (median Matthews Correlation Coefficient up to ∼0.79 vs. ∼0.51 for LASSO). On real datasets, SplitWise matched or slightly improved RMSE while selecting fewer predictors. For instance, on Wine Quality (White), it improved RMSE from 0.756 to 0.752 and on Wine Quality (Red) from 0.654 to 0.649, using 6–10 predictors. On Bodyfat, it achieved 3.48–3.49 RMSE with four predictors, comparable to Elastic
Net (3.41–3.48 RMSE) but with smaller models.
in regression modeling. We introduce SplitWise, a stepwise regression framework that adaptively transforms numeric predictors into threshold-based binary features using shallow decision trees—only when such transformations improve model fit according to the Akaike or Bayesian Information Criterion. This design preserves the transparency of linear models while flexibly capturing threshold-based nonlinear effects, positioning SplitWise between classical linear and interpretable nonlinear
regression. SplitWise retains a single, globally linear equation that selectively incorporates data-driven thresholds—yielding models that remain straightforward to interpret and verify. Across synthetic scenarios with nonlinear signal patterns, SplitWise reduced median RMSE by 7–14% relative to the best-performing interpretable linear baseline and improved variable-selection accuracy (median Matthews Correlation Coefficient up to ∼0.79 vs. ∼0.51 for LASSO). On real datasets, SplitWise matched or slightly improved RMSE while selecting fewer predictors. For instance, on Wine Quality (White), it improved RMSE from 0.756 to 0.752 and on Wine Quality (Red) from 0.654 to 0.649, using 6–10 predictors. On Bodyfat, it achieved 3.48–3.49 RMSE with four predictors, comparable to Elastic
Net (3.41–3.48 RMSE) but with smaller models.
Date Issued
2025-11-27
Date Acceptance
2025-10-29
Citation
Scientific Reports, 2025, 15
ISSN
2045-2322
Publisher
Springer Science and Business Media LLC
Journal / Book Title
Scientific Reports
Volume
15
Copyright Statement
© The Author(s) 2025 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Publication Status
Published
Article Number
42454
Date Publish Online
2025-11-27