Variability-aware request replication for latency curtailment
File(s)PID4064225.pdf (453.66 KB)
Accepted version
Author(s)
Harrison, PG
Qiu, Z
Perez, JF
Type
Conference Paper
Abstract
Processing time variability is commonplace in distributed systems, where resources display disparate performance due to, e.g., different workload levels, background processes, and contention in virtualized environments. However, it is paramount for service providers to keep variability in response time under control in order to offer responsive services. We investigate how request replication can be used to exploit processing time variability to reduce response times, considering not only mean values but also the tail of the response time distribution. We focus on the distributed setup, where replication is achieved by running copies of requests on multiple servers that otherwise evolve independently, and waiting for the first replica to complete service. We construct models that capture the evolution of a system with replicated requests using approximate methods and observe that highly variable service times offer the best opportunities for replication ¿¿¿ reducing the response time tail in particular. Further, the effect of replication is non-uniform over the response time distribution: gains in one metric, e.g., the mean, can be at the cost of another, e.g., the tail percentiles. This is demonstrated in wide range of numerical virtual experiments. It can be seen that capturing service time variability is key to the evaluation of latency tolerance strategies and in their design.
Date Issued
2016-04-10
Date Acceptance
2015-10-25
Citation
Proceedings of the 35th Annual IEEE International Conference on Computer Communications (IEEE INFOCOM 2016), 2016, pp.1-9
Publisher
IEEE
Start Page
1
End Page
9
Journal / Book Title
Proceedings of the 35th Annual IEEE International Conference on Computer Communications (IEEE INFOCOM 2016)
Copyright Statement
© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Sponsor
Commission of the European Communities
Engineering & Physical Science Research Council (EPSRC)
Grant Number
FP7 - 318484
EP/L00738X/1
Source
35th Annual IEEE International Conference on Computer Communications (IEEE INFOCOM 2016)
Publication Status
Published
Start Date
2016-04-10
Finish Date
2016-04-14
Coverage Spatial
San Francisco, CA, USA