Dear Spiral user, We’re pleased to introduce Helix, https://helix.imperial.ac.uk/ Imperial’s new repository for research datasets. Please use Helix for all future dataset deposits. There is no change to Spiral - continue depositing as usual. For queries, contact the Research Data Management team at rdm-enquiries@imperial.ac.uk. Kind regards, the Spiral Team. The Spiral team
Putting randomized compiler testing into production
File(s)
LIPIcs-ECOOP-2020-22.pdf (1.27 MB)
Published version
OA Location
Author(s)
Donaldson, AF
Evrard, H
Thomson, P
Type
Conference Paper
Abstract
We describe our experience over the last 18 months on a compiler testing technology transfer project: taking the GraphicsFuzz research project on randomized metamorphic testing of graphics shader compilers, and building the necessary tooling around it to provide a highly automated process for improving the Khronos Vulkan Conformance Test Suite (CTS) with test cases that expose fuzzer-found compiler bugs, or that plug gaps in test coverage. We present this tooling for test automation – gfauto – in detail, as well as our use of differential coverage and test case reduction as a method for automatically synthesizing tests that fill coverage gaps. We explain the value that GraphicsFuzz has provided in automatically testing the ecosystem of tools for transforming, optimizing and validating Vulkan shaders, and the challenges faced when testing a tool ecosystem rather than a single tool. We discuss practical issues associated with putting automated metamorphic testing into production, related to test case validity, bug de-duplication and floating-point precision, and provide illustrative examples of bugs found during our work.
Date Issued
2020-11-06
Date Acceptance
2020-11-01
Citation
Leibniz International Proceedings in Informatics, LIPIcs, 2020, 166, pp.1-29
URI
URL
DOI
ISBN
9783959771542
ISSN
1868-8969
Publisher
DROPS
Start Page
1
End Page
29
Journal / Book Title
Leibniz International Proceedings in Informatics, LIPIcs
Volume
166
Copyright Statement
© Alastair F. Donaldson, Hugues Evrard, and Paul Thomson;
licensed under Creative Commons License CC-BY
34th European Conference on Object-Oriented Programming (ECOOP 2020).
License URL
Identifier
https://drops.dagstuhl.de/opus/volltexte/2020/13179/
Source
34th European Conference on Object-Oriented Programming (ECOOP 2020)
Publication Status
Published
Start Date
2020-11-15
Finish Date
2020-11-17
Coverage Spatial
Berlin, Germany
Date Publish Online
2020-11-06