AT-GIS: highly parallel spatial query processing with associative transducers

File Description SizeFormat 
ogden2016_sigmod.pdfAccepted version264.52 kBAdobe PDFView/Open
Title: AT-GIS: highly parallel spatial query processing with associative transducers
Authors: Ogden
Thomas, D
Pietzuch, P
Item Type: Conference Paper
Abstract: Users in many domains, including urban planning, transportation, and environmental science want to execute analytical queries over continuously updated spatial datasets. Current solutions for largescale spatial query processing either rely on extensions to RDBMS, which entails expensive loading and indexing phases when the data changes, or distributed map/reduce frameworks, running on resource-hungry compute clusters. Both solutions struggle with the sequential bottleneck of parsing complex, hierarchical spatial data formats, which frequently dominates query execution time. Our goal is to fully exploit the parallelism offered by modern multicore CPUs for parsing and query execution, thus providing the performance of a cluster with the resources of a single machine. We describe AT-GIS, a highly-parallel spatial query processing system that scales linearly to a large number of CPU cores. ATGIS integrates the parsing and querying of spatial data using a new computational abstraction called associative transducers(ATs). ATs can form a single data-parallel pipeline for computation without requiring the spatial input data to be split into logically independent blocks. Using ATs, AT-GIS can execute, in parallel, spatial query operators on the raw input data in multiple formats, without any pre-processing. On a single 64-core machine, AT-GIS provides 3× the performance of an 8-node Hadoop cluster with 192 cores for containment queries, and 10× for aggregation queries.
Issue Date: Jun-2016
Date of Acceptance: 11-Nov-2015
DOI: 10.1145/2882903.2882962
Publisher: ACM
Replaces: 10044/1/30265
Copyright Statement: © 2016 ACM
Conference Name: ACM SIGMOD International Conference on Management of Data 2016
Keywords: Science & Technology
Computer Science, Information Systems
Computer Science
Publication Status: Published
Start Date: 2016-06-26
Finish Date: 2016-07-01
Conference Place: San Francisco, USA
Online Publication Date: 2016-06
Appears in Collections:Faculty of Engineering
Electrical and Electronic Engineering