Time to clean your test objectives
File(s)
Published version
Author(s)
Type
Conference Paper
Abstract
Testing is the primary approach for detecting software defects. A major challenge faced by testers lies in crafting efficient test suites, able to detect a maximum number of bugs with manageable effort. To do so, they rely on coverage criteria, which define some precise test objectives to be covered. However, many common criteria specify a significant number of objectives that occur to be infeasible or redundant in practice, like covering dead code or semantically equal mutants. Such objectives are well-known to be harmful to the design of test suites, impacting both the efficiency and precision of the tester's effort. This work introduces a sound and scalable technique to prune out a significant part of the infeasible and redundant objectives produced by a panel of white-box criteria. In a nutshell, we reduce this task to proving the validity of logical assertions in the code under test. The technique is implemented in a tool that relies on weakest-precondition calculus and SMT solving for proving the assertions. The tool is built on top of the Frama-C verification platform, which we carefully tune for our specific scalability needs. The experiments reveal that the pruning capabilities of the tool can reduce the number of targeted test objectives in a program by up to 27% and scale to real programs of 200K lines, making it possible to automate a painstaking part of their current testing process.
Date Issued
2018-05-27
Date Acceptance
2018-05-01
Citation
ICSE '18: Proceedings of the 40th International Conference on Software Engineering, 2018, pp.456-467
ISBN
9781450356381
Publisher
IEEE
Start Page
456
End Page
467
Journal / Book Title
ICSE '18: Proceedings of the 40th International Conference on Software Engineering
Replaces
10044/1/55595
Copyright Statement
© 2018 Copyright held by the owner/author(s).
Identifier
https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000454843300052&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=a2bf6146997ec60c407a63945d4e92bb
Source
40th ACM/IEEE International Conference on Software Engineering (ICSE)
Subjects
Computer Science
Computer Science, Software Engineering
Coverage Criteria
EQUIVALENT
EXPERIENCE
Infeasible Objectives
MUTATION
Redundant Objectives
Science & Technology
Technology
Publication Status
Published
Start Date
2018-05-27
Finish Date
2018-06-03
Coverage Spatial
SWEDEN, Gothenburg
Date Publish Online
2018-05-27