Verification-friendly networks: the case for parametric ReLUs
File(s)
Accepted version
Author(s)
Leofante, Francesco
Henriksen, Patrick
Lomuscio, Alessio
Type
Conference Paper
Abstract
It has increasingly been recognised that verification can contribute to the validation and debugging of neural networks before deployment, particularly in safety-critical areas. While progress has been made in the area of
verification of neural networks, present techniques still do not scale to large ReLU-based neural networks used in many applications. In this paper we show that considerable progress can be made by employing Parametric ReLU
activation functions in lieu of plain ReLU functions. We give training procedures that produce networks which achieve one order of magnitude gain in verification overheads and 30-100% fewer timeouts with VeriNet, a SoA Symbolic Interval Propagation-based verification toolkit, while not compromising the resulting accuracy. Furthermore, we show that adversarial training combined with our approach
improves certified robustness up to 36% compared to adversarial training performed on baseline ReLU networks.
verification of neural networks, present techniques still do not scale to large ReLU-based neural networks used in many applications. In this paper we show that considerable progress can be made by employing Parametric ReLU
activation functions in lieu of plain ReLU functions. We give training procedures that produce networks which achieve one order of magnitude gain in verification overheads and 30-100% fewer timeouts with VeriNet, a SoA Symbolic Interval Propagation-based verification toolkit, while not compromising the resulting accuracy. Furthermore, we show that adversarial training combined with our approach
improves certified robustness up to 36% compared to adversarial training performed on baseline ReLU networks.
Date Issued
2023-08-02
Date Acceptance
2023-04-07
Citation
2023, pp.1-9
Publisher
IEEE
Start Page
1
End Page
9
Copyright Statement
Copyright © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Identifier
https://ieeexplore.ieee.org/document/10191169
Source
International Joint Conference on Neural Networks (IJCNN 2023)
Publication Status
Published
Start Date
2023-06-18
Finish Date
2023-06-23
Coverage Spatial
Queensland, Australia
Date Publish Online
2023-08-02