IRUS Total

Metamorphic Testing for Software Libraries and Graphics Compilers

File Description SizeFormat 
Lascu-A-2022-PhD-Thesis.pdfThesis2.7 MBAdobe PDFView/Open
Title: Metamorphic Testing for Software Libraries and Graphics Compilers
Authors: Lascu, Andrei
Item Type: Thesis or dissertation
Abstract: Metamorphic Testing is a testing technique which mutates existing test cases in semantically equivalent forms, by making use of metamorphic relations, while avoiding the oracle problem. However, these required relations are not readily available for a given system under test. Defining effective metamorphic relations is difficult, and arguably the main obstacle towards adoption of metamorphic testing in production-level software development. One example application is testing graphics compilers, where the approximate and under-specified nature of the domain makes it hard to apply more traditional techniques. We propose an approach with a lower barrier of entry to applying metamorphic testing for a software library. The user must still identify relations that hold over their particular library, but can do so within a development-like environment. We apply methods from the domains of metamorphic testing and fuzzing to produce complex test cases. We consider the user interaction a bonus, as they can control what parts of the target codebase is tested, potentially focusing on less-tested or critical sections of the codebase. We implement our proposed approach in a tool, MF++, which synthesises C++ test cases for a C++ library, defined by user-provided ingredients. We applied MF++ to 7 libraries in the domains of satisfiability modulo theories and Presburger arithmetic,. Our evaluation of MF++ was able to identify 21 bugs in these tools. We additionally provide an automatic reducer for tests generated by MF++, named MF++R. In addition to minimising tests exposing issues, MF++R can also be used to identify incorrect user-provided relations. Additionally, we investigate the combined use of MF++ and MF++R in order to augment code coverage of library test suites. We assess the utility of this application by contributing 21 tests aimed at improving coverage across 3 libraries.
Content Version: Open Access
Issue Date: Nov-2021
Date Awarded: Apr-2022
URI: http://hdl.handle.net/10044/1/96979
DOI: https://doi.org/10.25560/96979
Copyright Statement: Creative Commons Attribution NonCommercial Licence
Supervisor: Donaldson, Alastair
Sponsor/Funder: EPSRC
Funder's Grant Number: EP/L016796/1
Department: Computing
Publisher: Imperial College London
Qualification Level: Doctoral
Qualification Name: Doctor of Philosophy (PhD)
Appears in Collections:Computing PhD theses

This item is licensed under a Creative Commons License Creative Commons