Experiments or simulation? A characterization of evaluation methods for in-memory databases

File Description SizeFormat 
camera_ready_pdfexpress.pdfAccepted version671.08 kBAdobe PDFView/Open
Title: Experiments or simulation? A characterization of evaluation methods for in-memory databases
Authors: Molka, K
Casale, G
Item Type: Conference Paper
Abstract: © 2015 IFIP.The recent growth of interest for in-memory databases poses the question on whether established prediction methods such as response surfaces and simulation are effective to describe the performance of these systems. In particular, the limited dependence of in-memory technologies on the disk makes methods such as simulation more appealing than in the past, since disks are difficult to simulate. To answer this question, we study an in-memory commercial solution, SAP HANA, deployed on a high-end server with 120 physical cores. First, we apply experimental design methods to generate response surfaces that describe database performance as a function of workload and hardware parameters. Next, we develop a class-switching queueing network model to predict in-memory database performance under similar scenarios. By comparing the applicability of the two approaches to modeling multi-tenancy, we find that both queueing and response surface models yield mean prediction errors in the range 5%-22% with respect to mean memory occupancy and response times, but the accuracy for the latter deteriorates in response surfaces as the number of experiments are reduced, whereas simulation is effective in all cases. This suggests that simulation can be very effective in performance prediction for in-memory database management.
Issue Date: 9-Nov-2015
Date of Acceptance: 18-Aug-2015
URI: http://hdl.handle.net/10044/1/30336
DOI: https://dx.doi.org/10.1109/CNSM.2015.7367360
ISBN: 978-3-9018-8278-4
Publisher: IEEE
Start Page: 201
End Page: 209
Journal / Book Title: Network and Service Management (CNSM), 2015 11th International Conference on
Copyright Statement: © 2015 IFIP
Sponsor/Funder: Commission of the European Communities
Funder's Grant Number: 644869
Conference Name: IEEE CNSM 2015
Publication Status: Published
Start Date: 2015-11-09
Finish Date: 2015-11-13
Conference Place: Barcelona
Appears in Collections:Faculty of Engineering

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Creative Commonsx