Sustained Petascale Performance of Seismic Simulations with SeisSol on SuperMUC

Seismic simulations in realistic 3D Earth models require peta- or even exascale compute power to capture small-scale features of high relevance for scientific and industrial applications. In this paper, we present optimizations of SeisSol -- a seismic wave propagation solver based on the Arbitrary high-order accurate DERivative (ADER) Discontinuous Galerkin method on fully adaptive, unstructured tetrahedral meshes -- to run simulations under production conditions at petascale performance. Improvements cover the entire simulation chain: from an enhanced ADER time integration via highly scalable routines for mesh input up to hardware-aware optimization of the innermost sparse-/dense-matrix kernels. Strong and weak scaling studies on the SuperMUC machine demonstrated up to 90% parallel efficiency and 45% floating point peak efficiency on 147k cores. For a simulation under production conditions (10^8 grid cells, 4.8*10^10 degrees of freedom, 5 seconds simulated time), we achieved a sustained performance of 1.09~PFLOPS.

@inproceedings{id1982,
  author = {Breuer, Alexander and Heinecke, Alexander and Rettenberger, Sebastian and Bader, Michael and Gabriel, Alice-Agnes and Pelties, Christian},
  booktitle = {International Supercomputing Conference (ISC) Proceedings},
  doi = {10.1007/978-3-319-07518-1\_1},
  language = {en},
  note = {PRACE ISC Award Winning Paper},
  pages = {1-18},
  publisher = {Springer International Publishing},
  series = {Lecture Notes in Computer Science},
  title = {Sustained Petascale Performance of Seismic Simulations with SeisSol on SuperMUC},
  url = {http://www5.in.tum.de/pub/isc14\_seissol\_supermuc.pdf},
  volume = {8488},
  year = {2014},
}

%O Conference Proceedings
%A Breuer, Alexander
%A Heinecke, Alexander
%A Rettenberger, Sebastian
%A Bader, Michael
%A Gabriel, Alice-Agnes
%A Pelties, Christian
%B International Supercomputing Conference (ISC) Proceedings
%R 10.1007/978-3-319-07518-1_1
%G en
%O PRACE ISC Award Winning Paper
%P 1-18
%I Springer International Publishing
%S Lecture Notes in Computer Science
%T Sustained Petascale Performance of Seismic Simulations with SeisSol on SuperMUC
%U http://www5.in.tum.de/pub/isc14_seissol_supermuc.pdf
%V 8488
%D 2014