Dreaming up metamorphic relations: experiences from three fuzzer tools
File(s)
Accepted version
Author(s)
Lascu, Andrei
Windsor, Matt
Donaldson, Alastair
Grosser, Tobias
Wickerson, John
Type
Conference Paper
Abstract
Metamorphic testing requires the availability of a suitable set of metamorphic relations (MRs) for the application domain of interest. A software testing practitioner interested in using metamorphic testing is thus blocked unless they can devise a suitable set of MRs. In this paper we offer some practical advice on sources of inspiration for MRs, based on our experience building three fuzzing tools based on metamorphic testing: MF++, which supports automated testing of C++11libraries,C4, which tests concurrency support in C11 compilers, and spirv-fuzz, which aims to find bugs in compilers for the SPIR-V programming language (mainly used in computer graphics).The MRs we have devised have taken inspiration from three main sources: (1) careful study of specification documents related to the libraries and programming languages that these tools target, (2) consultation of prior work and discussion with domain experts, and (3) manual inspection of the results of automated code coverage analysis on the systems under test. We describe these sources of inspiration in detail, giving a range of concrete examples for each. We hope that this experience report will help to inform developers of future metamorphic testing tools as to the steps they can take to discover MRs in their domains of interest.
Date Issued
2021-07-12
Date Acceptance
2021-02-22
Citation
2021 IEEE/ACM 6th International Workshop on Metamorphic Testing (MET), 2021, pp.61-68
Publisher
IEEE
Start Page
61
End Page
68
Journal / Book Title
2021 IEEE/ACM 6th International Workshop on Metamorphic Testing (MET)
Copyright Statement
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Engineering & Physical Science Research Council (E
Grant Number
EP/R011605/1
Ref: 542716
Source
2021 IEEE/ACM 6th International Workshop on Metamorphic Testing (MET)
Subjects
Science & Technology
Technology
Computer Science, Software Engineering
Computer Science, Theory & Methods
Computer Science
metamorphic relations
fuzzing
metamorphic testing
compiler testing
library testing
C11
SPIR-V
Publication Status
Published
Start Date
2021-06-02
Finish Date
2021-06-02
Coverage Spatial
Madrid, Spain