Portable shared-memory parallelisation strategies for high-order finite element codes
File(s)
Author(s)
Eichstädt, Jan Robert
Type
Thesis
Abstract
This thesis investigates strategies to parallelise numerical simulation software frameworks in the context of high-order finite element solvers. Due to the recent evolution of high performance computing (HPC) hardware from single-core central processing units (CPUs) to multi-core CPUs and the emergence of general purpose graphics processing units (GPGPUs or short GPUs), these shared memory systems are now characterised by a high degree of parallelism. They also possess a variety of architecture-specific features that must be appropriately utilised in order to fully realise their capabilities. Large-scale legacy solver frameworks thus face the question how to adapt their codebase in the most effective way, to enable performance and portability across a heterogeneous landscape of parallel HPC systems, while ensuring code maintainability.
Various approaches to address this challenge are considered using practical investigations of core numerical algorithms for high-order finite element methods in order to ensure representative performance profiles. The selected core algorithms are a parallel mesh-optimisation method that optimises elemental shapes by minimising a deformation energy, and an implicit solver of the Helmholtz equation which forms the computationally
most demanding part of a complete incompressible Navier-Stokes solver.
Overall the results presented here clearly demonstrate that algorithmic adaptations have to be introduced to enable execution on modern CPU and GPU systems and to
achieve reasonable performance. Key aspects for an efficient mapping of the algorithms onto the hardware are considered: colouring approaches in connection with reduction operations, alongside determining the most effective manner of utilising multi-level nested parallel algorithms on multi-level parallel shared-memory hardware. It is further shown that the layout of data structures and their placement in memory is often the most important aspect in improving performance for both CPU and GPU systems. In order to achieve efficient memory access over vectorised operations, an interleaved data layout is developed. In a novel approach for elemental evaluations that minimises memory bandwidth,
performance profiling demonstrates that on GPUs faster runtimes can be achieved, despite additional arithmetic operations. The balance between the utilisation of different hardware systems is assessed using runtimes, performance modelling, and a novel operational cost metric, and demonstrates that executions of the investigated algorithms on GPUs are both faster and more cost-effective than on CPUs for the problems under
consideration. Apart from the algorithmic adaptations, two programming paradigms are investigated:
the effectiveness and performance of a single codebase, augmented with a variety of of portable programming models is compared against the implementation of a small number of hardware-specific kernels in regions of computational hotspots. The analysis here suggests that although the considered portable programming models (OpenMP, OpenACC, and Kokkos) should be easier to maintain, robust compiler support for all hardware types and unambiguous interpretation of compiler directives is not always guaranteed. The investigated GPU-specific CUDA kernels are most performant and achieve two to three times faster runtimes than all other models. Further work is proposed to follow the second paradigm in order to efficiently parallelise large software frameworks for high-order flow solvers.
Various approaches to address this challenge are considered using practical investigations of core numerical algorithms for high-order finite element methods in order to ensure representative performance profiles. The selected core algorithms are a parallel mesh-optimisation method that optimises elemental shapes by minimising a deformation energy, and an implicit solver of the Helmholtz equation which forms the computationally
most demanding part of a complete incompressible Navier-Stokes solver.
Overall the results presented here clearly demonstrate that algorithmic adaptations have to be introduced to enable execution on modern CPU and GPU systems and to
achieve reasonable performance. Key aspects for an efficient mapping of the algorithms onto the hardware are considered: colouring approaches in connection with reduction operations, alongside determining the most effective manner of utilising multi-level nested parallel algorithms on multi-level parallel shared-memory hardware. It is further shown that the layout of data structures and their placement in memory is often the most important aspect in improving performance for both CPU and GPU systems. In order to achieve efficient memory access over vectorised operations, an interleaved data layout is developed. In a novel approach for elemental evaluations that minimises memory bandwidth,
performance profiling demonstrates that on GPUs faster runtimes can be achieved, despite additional arithmetic operations. The balance between the utilisation of different hardware systems is assessed using runtimes, performance modelling, and a novel operational cost metric, and demonstrates that executions of the investigated algorithms on GPUs are both faster and more cost-effective than on CPUs for the problems under
consideration. Apart from the algorithmic adaptations, two programming paradigms are investigated:
the effectiveness and performance of a single codebase, augmented with a variety of of portable programming models is compared against the implementation of a small number of hardware-specific kernels in regions of computational hotspots. The analysis here suggests that although the considered portable programming models (OpenMP, OpenACC, and Kokkos) should be easier to maintain, robust compiler support for all hardware types and unambiguous interpretation of compiler directives is not always guaranteed. The investigated GPU-specific CUDA kernels are most performant and achieve two to three times faster runtimes than all other models. Further work is proposed to follow the second paradigm in order to efficiently parallelise large software frameworks for high-order flow solvers.
Version
Open Access
Date Issued
2020-06
Date Awarded
2021-01
Copyright Statement
Creative Commons Attribution-Non Commercial 4.0 International Licence
License URL
Advisor
Peiró, Joaquim
Publisher Department
Aeronautics
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)