IRUS Total

THEMIS: Fairness in Federated Stream Processing under Overload

File Description SizeFormat 
themis-sigmod2016 (1).pdfAccepted version751 kBAdobe PDFView/Open
Title: THEMIS: Fairness in Federated Stream Processing under Overload
Authors: Kalyvianaki, E
Fiscato, M
Salonidis, T
Pietzuch, PR
Item Type: Conference Paper
Abstract: Federated stream processing systems, which utilise nodes from multiple independent domains, can be found increasingly in multi-provider cloud deployments, internet-of-things systems, collaborative sensing applications and large-scale grid systems. To pool resources from several sites and take advantage of local processing, submitted queries are split into query fragments, which are executed collaboratively by different sites. When supporting many concurrent users, however, queries may exhaust available processing resources, thus requiring constant load shedding. Given that individual sites have autonomy over how they allocate query fragments on their nodes, it is an open challenge how to ensure global fairness on processing quality experienced by queries in a federated scenario. We describe THEMIS, a federated stream processing system for resource-starved, multi-site deployments. It executes queries in a globally fair fashion and provides users with constant feedback on the experienced processing quality for their queries. THEMIS associates stream data with its source information content (SIC), a metric that quantifies the contribution of that data towards the query result, based on the amount of source data used to generate it. We provide the BALANCE-SIC distributed load shedding algorithm that balances the SIC values of result data. Our evaluation shows that the BALANCE-SIC algorithm yields balanced SIC values across queries, as measured by Jains Fairness Index. Our approach also incurs a low execution time overhead.
Issue Date: 26-Jun-2016
Date of Acceptance: 16-Nov-2015
URI: http://hdl.handle.net/10044/1/29611
DOI: http://dx.doi.org/10.1145/2882903.2882943
ISBN: 978-1-4503-3531-7
Publisher: ACM
Start Page: 541
End Page: 553
Journal / Book Title: Proceedings of the 2016 International Conference on Management of Data (SIGMOD '16)
Copyright Statement: © ACM 2016. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 2016 International Conference on Management of Data (SIGMOD '16), http://dx.doi.org/10.1145/2882903.2882943.
Conference Name: 2016 International Conference on Management of Data (SIGMOD '16)
Publication Status: Published
Start Date: 2016-06-26
Finish Date: 2016-07-01
Conference Place: San Francisco, CA, USA
Appears in Collections:Faculty of Engineering