Web technologies for environmental Big Data
File(s)Environmental Modelling & Software_63_2015.pdf (2.14 MB)
Published version
Author(s)
Vitolo, Claudia
Type
Journal Article
Abstract
Recent evolutions in computing science and web technology provide the environmental community with continuously expanding resources for data collection and analysis that pose unprecedented challenges to the design of analysis methods, workflows, and interaction with data sets. In the light of the recent UK Research Council funded Environmental Virtual Observatory pilot project, this paper gives an overview of currently available implementations related to web-based technologies for processing large and heterogeneous datasets and discuss their relevance within the context of environmental data processing, simulation and prediction. We found that, the processing of the simple datasets used in the pilot proved to be relatively straightforward using a combination of R, RPy2, PyWPS and PostgreSQL. However, the use of NoSQL databases and more versatile frameworks such as OGC standard based implementations may provide a wider and more flexible set of features that particularly facilitate working with larger volumes and more heterogeneous data sources.
Date Issued
2014-10-31
Citation
Environmental Modelling & Software, 2014, 63, pp.185-198
ISSN
1364-8152
Publisher
Elsevier
Start Page
185
End Page
198
Journal / Book Title
Environmental Modelling & Software
Volume
63
Copyright Statement
© 2014 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/3.0/).
License URL
Identifier
http://www.imperial.ac.uk/people/c.vitolo
Subjects
Web-based modelling
Big Data
Web services
OGC standards
Virtual Observatory
Publication Status
Published