Abstraction-aware inference of metamorphic relations
File(s)3643747.pdf (915.09 KB)
Published version
Author(s)
Type
Journal Article
Abstract
Metamorphic testing is a valuable technique that helps in dealing with the oracle problem. It involves testing software against specifications of its intended behavior given in terms of so called metamorphic relations, statements that express properties relating different software elements (e.g., different inputs, methods, etc). The effective application of metamorphic testing strongly depends on identifying suitable domain-specific metamorphic relations, a challenging task, that is typically manually performed. This paper introduces MemoRIA, a novel approach that aims at automatically identifying metamorphic relations. The technique focuses on a particular kind of metamorphic relation, which asserts equivalences between methods and method sequences. MemoRIA works by first generating an object-protocol abstraction of the software being tested, then using fuzzing to produce candidate relations from the abstraction, and finally validating the candidate relations through run-time analysis. A SAT-based analysis is used to eliminate redundant relations, resulting in a concise set of metamorphic relations for the software under test. We evaluate our technique on a benchmark consisting of 22 Java subjects taken from the literature, and compare MemoRIA with the metamorphic relation inference technique SBES. Our results show that by incorporating the object protocol abstraction information, MemoRIA is able to more effectively infer meaningful metamorphic relations, that are also more precise, compared to SBES, measured in terms of mutation analysis. Also, the SAT-based reduction allows us to significantly reduce the number of reported metamorphic relations, while in general having a small impact in the bug finding ability of the corresponding obtained relations.
Date Issued
2024-07
Date Acceptance
2024-01-23
Citation
Proceedings of the ACM on Software Engineering, 2024, 1 (FSE), pp.450-472
ISSN
2994-970X
Publisher
Association for Computing Machinery (ACM)
Start Page
450
End Page
472
Journal / Book Title
Proceedings of the ACM on Software Engineering
Volume
1
Issue
FSE
Copyright Statement
© 2024 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution 4.0 International License.
License URL
Identifier
http://dx.doi.org/10.1145/3643747
Publication Status
Published
Article Number
21
Date Publish Online
2024-07-12