Global optimization of gaussian process acquisition functions using a piecewise-linear kernel approximation
File(s)
Published version
Author(s)
Xie, Y
Zhang, S
Paulson, JA
Tsay, C
Type
Conference Paper
Abstract
Bayesian optimization relies on iteratively constructing and optimizing an acquisition function. The latter turns out to be a challenging, non-convex optimization problem itself. Despite the relative importance of this step, most algorithms employ sampling- or gradient-based methods, which do not provably converge to global optima. This work investigates mixed-integer programming (MIP) as a paradigm for global acquisition function optimization. Specifically, our Piecewise-linear Kernel Mixed Integer Quadratic Programming (PK-MIQP) formulation introduces a piecewise-linear approximation for Gaussian process kernels and admits a corresponding MIQP representation for acquisition functions. The proposed method is applicable to uncertainty-based acquisition functions for any stationary or dot-product kernel. We analyze the theoretical regret bounds of the proposed approximation, and empirically demonstrate the framework on synthetic functions, constrained benchmarks, and a hyperparameter tuning task.
Date Issued
2025-05-03
Date Acceptance
2025-05-01
Citation
Proceedings of Machine Learning Research, 2025, 258, pp.2296-2304
Publisher
PMLR
Start Page
2296
End Page
2304
Journal / Book Title
Proceedings of Machine Learning Research
Volume
258
Copyright Statement
Copyright © 2025 by the author(s).
Source
International Conference on Artificial Intelligence and Statistics
Publication Status
Published
Start Date
2025-05-03
Finish Date
2025-05-05
Coverage Spatial
Mai Khao, Thailand