SI2-SSI: A Sustainable Community Software Framework for Petascale Earthquake ModelingThomas H. Jordan (USC), Jacobo Bielak (CMU), Yifeng Cui (UCSD), Kim B. Olsen (SDSU)

S C E Can NSF+USGS center

EarthquakeEngineeringGeoscience

ComputerScience

SEISMSimulation Framework Valid

ation

Fra

mew

ork

Validation Exercises

Horizontal Integration

Vertical Integration

km

1

0.5

0

-0.5

-1

The main goal of the SEISM project is to advance and integrate scientific software elements developed by SCEC into a sustainable software environment for integrated seismic modeling (SEISM) as a software ecosystem for physics-based seismic hazard analysis (SHA).

The SEISM software ecosystem includes a collection of seismic hazard analysis software used by earthquake scientists, engineers, and risk managers to create the simulation-based products needed to improve SHA. The SEISM software provides the earthquake modeling community with high-performance computational tools that include model formulation, verification, prediction and validation, within a sustainable software engineering framework.

The CyberShake Seismic Hazard Model Platform:Documentation at: http://scec.usc.edu/scecpedia/CyberShakeSCEC’s CyberShake software platform utilizes 3D simulations and finite-fault rupture descriptions to compute probabilistic seismic hazards estimates for Southern California. CyberShake computational demands are intense, requiring parallel algorithms and high throughput workflows. Broad impact users of seismic hazard estimates, such as the USGS and civil engineers, are interested in using and extending the CyberShake computational system be-cause the results show long period ground motion effects such as coupling of directivity and basin response that cannot be captured with existing GMPE-based approaches. Recent CyberShake seismic hazard calculations have been extended to produce Risk-Targeted Maximum Considered Earthquake (MCER) ground motion estimates for potential use in Civil Engineering applications.

The Broadband Platform:Software Available at: http://scec.usc.edu/scecpedia/CyberShake

The Broadband platform generates 0-10 Hz ground motions using deterministic low-frequency and stochastic high-frequency simulations. The Broadband platform integrates different scientific software elements includ-ing rupture generation, low-frequency deterministic seismogram synthesis, high-frequency stochastic seis-mogram synthesis, and non-linear site effects. These complex scientific codes have been integrated into a system that supports easy on-demand computation of broadband seismograms. The Broadband platform is designed to be used by both geoscientists and engineers with some experience interpreting ground motion simulations.

Unified Community Velocity Model (UCVM) Platform:Software Available at: http://scec.usc.edu/scecpedia/UCVM

Hercules is an octree-based parallel finite element (FE) earthquake simulator developed by the Quake Group at Carn-egie Mellon, the University of Memphis, and SCEC. Hercules is written in C using MPI inter-processor communication libraries. It integrates an efficient unstructured hexahedral mesh generator and an explicit finite element formulation to solve the viscoelastodynamics equation for 3D wave propagation problems in highly heterogeneous media due to earthquake sources modeled with kinematic faulting. Alternative distribution of Hercules solve wave propagation problems in nonlinear elasto-plastic media and can account for surface topography and the presence of the built en-vironment. A recently developed version of Hercules uses C++ and CUDA for use of GPUs on hybrid systems.

The SEISM Project is supported by the National Science Foundation, award No. ACI-1148493. Additional SCEC team members include: J.W. Baker, G.C. Beroza (Stanford); P. Chen (U. of Wyoming); E.M. Dunham (Stanford); S.M. Day (SDSU); R.W. Graves (USGS); N. Luco (USGS); P.J. Maechling (USC); R. Taborda (CMU). Private and international partners include: Pacific Gas & Electric Co. and the REAKT European project. Scientific publications and ad-ditional information about the various software elements and platforms can be found at http://scec.usc.edu/scecpedia and at www.scec.org.

SCEC’s Anelastic Wave Propagation (AWP-ODC) high performance earthquake simulation software simu-lates the dynamic rupture and wave propagation that occurs during an earthquake. Dynamic rupture pro-duces friction, traction, slip, and slip rate information on the fault. The moment function is constructed from this fault data and used to initialize wave propagation. AWP-ODC is a well-established high performance sci-entific software that can perform determnistic ground motion simulations using forward and reciprocal tech-niques, with both CPU and GPU implementations. Recent physics-based improvements to AWP-ODC in-clude frequency dependent attenuation via a power law above a reference frequency, and off-fault visco-plastic yielding.

Hercules Finite Element Earthquake Simulation Software:Software Available at: https://github.com/CMU-Quake/hercules

AWP-ODC Finite Difference Earthquake Simulation Software:Documentation Available at: http://hpgeoc.sdsc.edu/AWPODC/

The SCEC Unified Community Velocity Model (UCVM) software framework is a collection of software tools and application programming interfaces designed to provide standardized access to multiple seismic ve-locity models used in seismology and geophysics research. Seismic velocity models are key components of current research efforts dedicated to advancing our knowledge of the Earth’s crustal structure and its influ-ence on ground response during earthquakes, including regional deep geology and local effects produced by the geometry, spatial distribution, and material composition of sediments in basins and valleys.

Figure 1: The SEISM Software ecosystems contains a collection of com-patible, related scientific applications that support physics-based seis-mic hazard analysis using high performance computing.

Figure 2: SEISM software ecosystem supports the “spiral of inference”, an in-terative process for developing valid physics-based software simulations.

Figure 3: The growing impact of the Broadband Platform (BBP) is indicated by (left) a set of 7 scientific and technical papers in Seismological Research Letters (Jan/Feb 2015) describing the BBP software and its use in a geoscientific and engineering ground motion simulation evaluation activity, and (right) use of the BBP by an international graduate student ground motion simulation training workshop in Sept 2013.

Figure 4: UCVM users can select a region of interest and then query UCVM to obtain material properties and create discrete three-dimensional grids or meshes of the region including meshes that exceed 100 billion pts.

Figure 5: Model of the Northridge earthquake with rough-fault topography, composed of 1.7 million subfaults that follow a self-similar fractal distribution with wavelengths larger than 80 m used as a source descption in high frequency AWP-ODC simulations that evaluated recent implementations of frequency depedent Q and use of small scale heterogeneities in seismic velocity models.

Figure 6: Example use of Her-cules in engineering-oriented ground motion study show-ing, from left to right (a) com-parative input seismic veloc-ity models, (b) Hercules output simulated ground mo-tions, (c) comparison of simu-lation results to observed ground motion data, and (d) comparision of simulated ground motion results with well-established engineering metrics.

Figure 7: Two CyberShake hazard models for the Los Angeles region calculated in 2014 on Blue Waters using a simple 1D earth model (left) and a more realistic 3D earth model (right). Seismic hazard estimates produced using the 3D earth model show lower near-fault intensities due to 3D scattering (1), much higher intensities in near-fault basins (2), higher intensities in the Los Angeles basins (3), and lower intensities in hard-rock areas (4).

SEISM Project Goals: More at: http://scec.usc.edu/scecpedia/SEISM_Project

ScientificSoftwareElements

Users:

Reinsurance

USGS

SCEC

EERCsNEES

RiskManagement

EarthquakeEngineering

Insurance

EmergencyPreparedness

FEMA

EarthScope

Public GTLTera3DPseudo-

dynamic

UCVM

CyberShake

Broadband

SEISM-IO Work-flows

Hercules

AWP-ODC

SORD

SWP

OpenSHA

OpenSees