SQPR: stream query planning with reuse
File(s)
Published version
Author(s)
Kalyvianaki, Evangelia
Wiesemann, Wolfram
Vu, Quang Hieu
Kuhn, Daniel
Pietzuch, Peter
Type
Report
Abstract
When users submit new queries to a distributed
stream processing system (DSPS), a query planner must allocate
physical resources, such as CPU cores, memory and network
bandwidth, from a set of hosts to queries. Allocation decisions
must provide the correct mix of resources required by queries,
while achieving an efficient overall allocation to scale in the
number of admitted queries. By exploiting overlap between
queries and reusing partial results, a query planner can conserve
resources but has to carry out more complex planning decisions.
In this paper, we describe SQPR, a query planner that targets
DSPSs in data centre environments with heterogeneous resources.
SQPR models query admission, allocation and reuse as a single
constrained optimisation problem and solves an approximate version
to achieve scalability. It prevents individual resources from
becoming bottlenecks by re-planning past allocation decisions
and supports different allocation objectives. As our experimental
evaluation in comparison with a state-of-the-art planner shows
SQPR makes efficient resource allocation decisions, even with a
high utilisation of resources, with acceptable overheads.
stream processing system (DSPS), a query planner must allocate
physical resources, such as CPU cores, memory and network
bandwidth, from a set of hosts to queries. Allocation decisions
must provide the correct mix of resources required by queries,
while achieving an efficient overall allocation to scale in the
number of admitted queries. By exploiting overlap between
queries and reusing partial results, a query planner can conserve
resources but has to carry out more complex planning decisions.
In this paper, we describe SQPR, a query planner that targets
DSPSs in data centre environments with heterogeneous resources.
SQPR models query admission, allocation and reuse as a single
constrained optimisation problem and solves an approximate version
to achieve scalability. It prevents individual resources from
becoming bottlenecks by re-planning past allocation decisions
and supports different allocation objectives. As our experimental
evaluation in comparison with a state-of-the-art planner shows
SQPR makes efficient resource allocation decisions, even with a
high utilisation of resources, with acceptable overheads.
Date Issued
2010-01-01
Citation
Departmental Technical Report: 10/11, 2010, pp.1-13
Publisher
Department of Computing, Imperial College London
Start Page
1
End Page
13
Journal / Book Title
Departmental Technical Report: 10/11
Copyright Statement
© 2010 The Author(s). This report is available open access under a CC-BY-NC-ND (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Publication Status
Published
Article Number
10/11