Software Performance Engineering using Virtual Time Program Execution
Author(s)
Baltas, Nikolaos
Type
Thesis
Abstract
In this thesis we introduce a novel approach to software performance engineering that is based
on the execution of code in virtual time. Virtual time execution models the timing-behaviour
of unmodified applications by scaling observed method times or replacing them with results
acquired from performance model simulation. This facilitates the investigation of "what-if" performance predictions of applications comprising an arbitrary combination of real code and
performance models. The ability to analyse code and models in a single framework enables
performance testing throughout the software lifecycle, without the need to to extract performance
models from code. This is accomplished by forcing thread scheduling decisions to take
into account the hypothetical time-scaling or model-based performance specifications of each
method. The virtual time execution of I/O operations or multicore targets is also investigated.
We explore these ideas using a Virtual EXecution (VEX) framework, which provides performance
predictions for multi-threaded applications. The language-independent VEX core is
driven by an instrumentation layer that notifies it of thread state changes and method profiling events; it is then up to VEX to control the progress of application threads in virtual time on top of the operating system scheduler. We also describe a Java Instrumentation Environment
(JINE), demonstrating the challenges involved in virtual time execution at the JVM level.
We evaluate the VEX/JINE tools by executing client-side Java benchmarks in virtual time
and identifying the causes of deviations from observed real times. Our results show that VEX
and JINE transparently provide predictions for the response time of unmodified applications
with typically good accuracy (within 5-10%) and low simulation overheads (25-50% additional
time). We conclude this thesis with a case study that shows how models and code can be
integrated, thus illustrating our vision on how virtual time execution can support performance
testing throughout the software lifecycle.
on the execution of code in virtual time. Virtual time execution models the timing-behaviour
of unmodified applications by scaling observed method times or replacing them with results
acquired from performance model simulation. This facilitates the investigation of "what-if" performance predictions of applications comprising an arbitrary combination of real code and
performance models. The ability to analyse code and models in a single framework enables
performance testing throughout the software lifecycle, without the need to to extract performance
models from code. This is accomplished by forcing thread scheduling decisions to take
into account the hypothetical time-scaling or model-based performance specifications of each
method. The virtual time execution of I/O operations or multicore targets is also investigated.
We explore these ideas using a Virtual EXecution (VEX) framework, which provides performance
predictions for multi-threaded applications. The language-independent VEX core is
driven by an instrumentation layer that notifies it of thread state changes and method profiling events; it is then up to VEX to control the progress of application threads in virtual time on top of the operating system scheduler. We also describe a Java Instrumentation Environment
(JINE), demonstrating the challenges involved in virtual time execution at the JVM level.
We evaluate the VEX/JINE tools by executing client-side Java benchmarks in virtual time
and identifying the causes of deviations from observed real times. Our results show that VEX
and JINE transparently provide predictions for the response time of unmodified applications
with typically good accuracy (within 5-10%) and low simulation overheads (25-50% additional
time). We conclude this thesis with a case study that shows how models and code can be
integrated, thus illustrating our vision on how virtual time execution can support performance
testing throughout the software lifecycle.
Date Issued
2013-01
Date Awarded
2013-07
Copyright Statement
Attribution NoDerivatives 4.0 International Licence (CC BY-ND)
Advisor
Field, Anthony
Kelly, Paul
Publisher Department
Computing
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)