Dear Spiral user, We’re pleased to introduce Helix, https://helix.imperial.ac.uk/ Imperial’s new repository for research datasets. Please use Helix for all future dataset deposits. There is no change to Spiral - continue depositing as usual. For queries, contact the Research Data Management team at rdm-enquiries@imperial.ac.uk. Kind regards, the Spiral Team. The Spiral team
GRANNY: Granular management of compute-intensive applications in the cloud
File(s)
nsdi25-segarra.pdf (2.07 MB)
Published version
OA Location
Author(s)
Segarra, C
Shillaker, S
Li, G
Mappoura, E
Bruno, R
Type
Conference Paper
Abstract
Parallel applications are typically implemented using multi-threading (with shared memory, e.g., OpenMP) or multiprocessing (with message passing, e.g., MPI). While it seems attractive to deploy such applications in cloud virtual machines (VMs), existing cloud schedulers fail to manage such applications efficiently: they cannot scale multi-threaded applications dynamically when more CPU cores in a VM become available, and they cause fragmentation over time due to the static allocation of multi-process applications to VMs. We describe GRANNY, a new distributed runtime that enables the fine-granular management of multi-threaded/process applications in cloud environments. GRANNY supports the vertical scaling of multi-threaded applications within a VM and the horizontal migration of multi-process applications between VMs. GRANNY achieves both through a single WebAssembly-based execution abstraction: Granules can execute application code with thread or process semantics and allow for efficient snapshotting. GRANNY scales up applications by adding more Granules, and de-fragments applications by migrating Granules between VMs. In both cases, it launches new Granules from snapshots efficiently. We evaluate GRANNY with dynamic scheduling policies and show that, compared to current schedulers, it reduces the makespan for OpenMP workloads by up to 60% and the fragmentation for MPI workloads by up to 25%.
Date Issued
2025-04-01
Date Acceptance
2025-04-01
Citation
Proceedings of the 22nd Usenix Symposium on Networked Systems Design and Implementation Nsdi 2025, 2025, pp.205-218
URI
ISBN
978-1-939133-46-5
Start Page
205
End Page
218
Journal / Book Title
Proceedings of the 22nd Usenix Symposium on Networked Systems Design and Implementation Nsdi 2025
Copyright Statement
© 2025 The Author(s). Published by Usenix Association.
Source
22nd USENIX Symposium on Networked Systems Design and Implementation
Publication Status
Published
Start Date
2025-04-28
Finish Date
2025-04-30
Coverage Spatial
Philadelphia, PA, USA