1. The Broadband CyberShake Platform: Improving Seismic Hazard Analysis using USC HPCC
Scott Callaghan
Southern California Earthquake Center
University of Southern California
SC11

2. Probabilistic Seismic Hazard Analysis
Builders ask seismologists: “What will the peak ground motion be at my new building in the next 50 years?”
Seismologists answer this question using Probabilistic Seismic Hazard Analysis (PSHA)
PSHA results used in building codes, insurance
California building codes impact billions of dollars of construction yearly
SCEC uses CyberShake platform to perform PSHA

3. Probability of exceeding 0.1g in 50 yrs
PSHA Reporting
PSHA information is relayed through
Hazard curves (for 1 location)
Hazard maps (for a region)
Curve for USC
Probability of exceeding 0.1g in 50 yrs

4. PSHA Methodology Pick a location of interest.
Define what future earthquakes might happen.
Estimate the magnitude and probability for each earthquake, from earthquake rupture forecast (ERF)
Determine the shaking caused by each earthquake at the site of interest.
Aggregate the shaking levels with the probabilities to produce a hazard curve.
Repeat for multiple sites for a hazard map.
Typically performed with attenuation relationships. 4

5. CyberShake Computations
Wave propagation simulation
Create 1.5 billion point mesh with material properties
Generate Strain Green Tensors (SGTs) for volume
Describe stresses and strains
Parallel, ~12,000 CPU-hrs

6. Second Phase Computations
Individual earthquake contributions
Use “seismic reciprocity” to simulate seismograms for each of 400,000 earthquakes
Calculate peak shaking, combine for hazard curve
Loosely-coupled, short-running serial jobs

7. High Frequencies CyberShake calculated seismograms up to 0.5 Hz
Higher frequencies influence shorter buildings
1-story building ~ 10 Hz
Higher frequency calculations are more intensive
2x frequency = 16x computational effort
Instead, use stochastic high frequency approach
Already in SCEC Broadband Platform
Use to compute ground motion Hz
Created Broadband CyberShake
Ran at USC HPCC

8. Computational Requirements (for 1 site)
Component
Data
Executions
Cores/exec
CPU hours
Mesh generation
15 GB 1
160
150
SGT simulation
40 GB 2
400
12,000
SGT extraction
690 GB
7,000
250
Low-frequency seismograms
10 GB
415,000
1,600
High-frequency seismograms
800
PSA calculation
90 MB
100
Curve generation
1 MB
< 1
Total
795 GB
1,252,000
14,900
SGT Creation
Post Processing

9. Scientific Workflow Tools
Pegasus
Create abstract workflow description (DAX)
Logical names, dependencies
Plan workflow for site-specific execution (DAG)
Logical names resolved
Adds stage-in and stage-out of data
Condor
Workflow submitted to DAGMan
Manages workflow execution on remote resources
Globus
Sends jobs across the grid
GridFTP for fast file transfer

10. Other CyberShake changes
New rupture realizations
More complexity
Less coherence
Hazard curves with previous (open circle) and new (closed circle) ruptures
Plots of slip for previous (top) and new (bottom) ruptures

11. Multiple Velocity Models
CyberShake now supports multiple velocity models
CVM-H minus CVM-S4, depth 0.0m
CVM-H 11.2 (red) vs CVM-S4 (blue)

12. Perris precariously balanced rock,1 s SA
Broadband results
interchange, 0.1 s SA
Perris precariously balanced rock,1 s SA
Currently calculating PBR and seismic stations for validation

13. Thanks!