Recent Achievements of the Collaboratory for the Study of Earthquake Predictability

● Establish rigorous procedures for registering and evaluatingearthquake forecasting and prediction experiments.

● Construct community standards and protocols for comparativetesting of earthquake predictions.

● Develop an infrastructure that allows groups of researchers toparticipate in prediction experiments.

● Provide access to authorized data sets and monitoringproducts for calibrating and testing earthquake predictionalgorithms.

● Accommodate experiments involving fault systems in differentgeographic and tectonic environments.

CSEP Software is released under GPL licenses and isdistributed to other earthquake forecast testing facilities outside ofCalifornia. We use Yellowdog Updater, Modified (YUM) repositorydistribution and Kernel-based Virtual Machine (KVM) images tokeep all testing centers automatically up-to-date to the currentversion of testing center codes. We host the XML-definitions, thesoftware core, and the scripts for testing regions.

Objectives● Promote rigorous research on earthquake predictability

through the SCEC program and its global partnerships.● Help the responsible government agencies assess the

feasibility of earthquake prediction and the performance ofproposed prediction algorithms.

● Reduce the controversy surrounding earthquake predictionthrough a collaboratory infrastructure to support a wide rangeof scientific prediction experiments.

Goals

D. D. Jackson1, M. Werner2, A. Christophersen5, P. Maechling3, D. Rhoades5, D. Schorlemmer3,4, A. Strader4, J. Yu3, W. Marzocchi6, T.H. Jordan3, and the CSEP Working Group1 Department of Earth, Planetary, and Space Science, UCLA, Los Angeles, USA.2 School of Earth Sciences and Cabot Institute, University of Bristol, UK.3 Southern California Earthquake Center, University of Southern California, Los Angeles, USA.4 Department 2: Physics of the Earth, German Research Centre for Geosciences, Potsdam, Germany.5 GNS Science, Lower Hutt, New Zealand.6 INGV, Rome, Italy.

Development organization

CSEP Software Automatic:● Catalog retrieval from authorized data source● Forecast generation and evaluation● Publishing of resultsCSEP software capabilities:● Reproduce any CSEP experiment● Reprocess forecasts and evaluations in batch mode Dataset archive:● Raw and post-processed catalogs● Input parameters for forecasts models, forecasts● Evaluation tests results

Mini CSEP Distribution For use by earthquake prediction researchers:● To retrieve catalogs from authorized data sources● To run evaluation tests for forecasts● To create spatial forecast distribution maps http://northridge.usc.edu/trac/csep/wiki/MiniCSEP

http://cseptesting.org

CSEP Testing Centers Models

Web Presentation

Figure 1. As of April 2016, there is the total of 438 models under test in CSEP

Global 0.1 Degree Testing Region

SCEC Testing Center Results

Please visit our web page http://cseptesting.org/results.

GEAR1: The first version of the GlobalEarthquake Activity Rate Model is anoptimized combination of a globalsmoothed seismicity model developed byKagan and Jackson (2011) using shallowmw ≥ 5.767 historical seismicity from theCMT earthquake catalog, and a tectonicmodel (Bird and Kreemer, 2015) based onglobal strain rates from version 2.1 of theG l o b a l S t r a i n R a t e M a p(SHIFT_GSRM2F). Using a loglinearcombination of the two parent forecasts,the GEAR1 model was optimized usingseismicity from 2005-2012.

SHIFT_GSRM: The tectonic earthquakeforecast was developed from the firstversion of the Global Strain Rate Model(GSRM). Strain rates, inferred from GPSvelocities and plate tectonics, wereconverted to long-term seismic momentrates (Kreemer et al., 2003) based on theshear modulus and tectonic regions'seismogenic depths. Earthquake rateswere derived using the corner magnitudes a n d s p e c t r a l s l o p e s o f t h efrequency/moment relation correspondingto each tectonic region.

SHIFT_GSRM2F: GSRM2.1 is anupdate of the SHIFT_GSRM model,constrained by four times the number ofgeodetic velocities. Additionally, the gridresolution was increased, with six timesthe number of cells as SHIFT_GSRM. Thenumber of (assumed) plates and blockswas doubled, to account for regions whererigidity can be assumed but GPScoverage is insufficient to adequatelyc o n s t r a i n b l o c k b o u n d a r i e s a n ddeformation zones.

Figure 1: Forecasted numbers of 6.0 ≤ mw ≤ 9.0 earthquakes in0.1˚x0.1˚x70km bins from 1. October 2015 until 1. March 2017 for theGEAR1 (top), SHIFT_GSRM (center) and SHIFT_GSRM2F (bottom)forecasts. Red circles indicate earthquake locations during the evaluation period, and are scaled to indicate relative magnitudes. Within the Pacificsubduction zone, forecasted seismicity rates are considerably greater forthe GEAR1 model than for the GSRM models, consistent with theabundance of earthquakes occurring in this region.

CSEP Comparative-Test Results

The T- and W-tests indicate that both SHIFT_GSRM and SHIFT_GSRM2F can be rejected in favor of the GEAR1 model at the 0.05 significance level. This result is consistent with the informationscores used to optimize the GEAR1 forecast from 2005-2012 (Bird et al., 2015), which indicate anoptimal loglinear combination of 60% from the smoothed seismicity forecast, and 40% from theSHIFT_GSRM2F model. There were no observed anomalous differences in log-likelihood scoresbetween the GEAR1 and GSRM models caused by specific events.

Figure 2: T- and W-test results during the one-year evaluation period. The plotsdisplay the average information gain per earthquake of the GSRM models over theGEAR1 model. Average information gain is indicated by the circles, while thehorizontal lines show the ranges of average information gain within the 0.05significance level. The numbers above the circles show the number of earthquakesoccurring during the testing period. Asterisks indicate that a forecast can be rejectedin favor of GEAR1 according to the W-test. Both GSRM models can be rejected infavor of GEAR1.

CSEP Consistency-Test ResultsN-Test (Number of Earthquakes)

L-Test (Data Consistency) All forecasts pass the L-test, indicatingoverall consistency between observedand forecasted earthquake distributions.Although the L-test provides informationabout the consistency of the number ofearthquakes, magnitude and spatialdistributions, the total log-likelihood score(L) is most influenced by the number offorecasted earthquakes. Therefore, littleinformation beyond the N-test results isapparent from the L-test.

CL-Test (Data Consistency withNormalized Earthquake Count)

The number of earthquakes forecasted by all three models is consistent withobserved seismicity for the one-yearprospective evaluation period. As theduration of the evaluation periodincreases, forecasted earthquakenumbers approach the observedearthquake count.

M-Test (Magnitude Distribution)

S-Test (Spatial Distribution) All of the forecasts fail the S-test, whichevaluates the consistency of theforecasts' spatial distributions with thatof observed seismicity. For both GSRMforecasts, the log-likelihood score fromobserved earthquakes falls below therange considered consistent with theforecasts. In the case of the GEAR1forecast, the observed log-likelihoodscore exceeds the forecasted range.Because in both cases the forecastedand observed log-likelihood scores areinconsistent, the S-test (as well as alllikelihood consistency tests) wereconducted as two-tailed tests.Figure 3: CSEP consistency test results for the

one-year prospective evaluation period. Thesquares indicate observed numbers of earthquakes (N-test) or log-likelihood scores (all other tests).Green squares indicate the observed earthquakenumber or log-likelihood score is consistent withwhat would be simulated from the forecasts. Thehorizontal lines indicate the range of simulatedearthquake numbers or log-likelihood scores withinthe 0.05 significance level.

All three forecasts pass the M-testduring the one-year evaluation period,i n d i c a t i n g t h a t t h e m a g n i t u d edistribution of observed earthquakesduring the one-year evaluation period is cons is ten t w i th the fo recas tedmagnitude distribution. The forecastedmagnitude distributions were obtainedfrom the union of various taperedGutenberg-Richter distributions fordifferent tectonic regions.

The conditional likelihood (CL) testcompares the spat ia l -magni tudedistribution of observed versus synthetic earthquake catalogs by normalizing thenumber of forecasted events to beequivalent with the observed number.During the evaluation period, only theSHIFT_GSRM forecast passes the CL-test. The other two forecasts fail the CL-test due to inconsistencies in theforecasted versus observed spatialearthquake distribution (see S-testresults).

Testing Fault-based Earthquake Models

Earthquakes on faults play an important role in hazard studies, yet models of finiteruptures on faults are difficult to test for several reasons.

● Fault ruptures are subjective and lack systematic reporting in authoritative catalogs.● Surface observations may not reveal the true extent of subsurface rupture, and● Surface ruptures are relatively rare, and earthquakes observed seismically and bysurface rupture are too rare to provide adequate statistics. ● Earthquakes may occur off of but near previously mapped faults, making forecastsdifficult to state in a testable fashion.

Here we explore ways of testing Uniform California Earthquake Rupture Forecasts(UCERF) both retrospectively and prospectively.

Key features include● Specifying polygons about fault sections, such that earthquakes not precisely on the faultbut within the polygons affect the stress and recurrence properties on the fault, and● Determining rupture extent by seismically observed aftershock locations, and ● Defining goodness of fit by whether rupture stops or continues through test polygonsalong faults with prescribed conditional probability that rupture will stop within a polygon ifit enters.

We test probabilities based on hypotheses (e.g. segmentation) assumed in the recurrencemodel against those based on a null hypothesis that earthquakes occur on an infinitelylong fault with a length distribution based on Gutenberg-Richter size distribution andsimple magnitude-length scaling..

Figure 4: Map of seismio-tectoniczones used in the 1995 WGCEPforecast, and M3+ aftershocks within 30days of the four subsequent M6+earthquakes. Probabilities were assignedto each of the zones in 1995. The figureshows how rupture extent can be definedusing aftershock epicenters.

Figure 6: Conditional stoppingprobabilities as a function of distancesouthward from Mendocino for the SanAndreas Fault (adapted from Figure 29of Field et al., 2014). Red curve is basedon modeled earthquake ruptures in theUCERF3 report, while the blue curveillustrates the null hypothesis. Both areevaluated on about 5 km sub-segments.

Figure 5: Conditional stoppingprobabilities for 20 km fault segmentboundary zones on the southern SanAndreas fault. Blue bars are calculatedfrom segment rupture forecasts inUCERF2 (Field et al, 2009); horizontalred line is for null hypothesis of randomlyplaced self-similar GR ruptures oninfinitely long fault.

Present efforts include mapping rupture zones as illustrated in Figure 4 onto thestopping probabilities in Figure 3. The UCERF3 model implies relevant earthquakes alongthe San Andreas fault at a rate of about 0.15 per year. A definitive prospective test mighttake several decades, but likelihood scores from just one large earthquake might falsify the UCERF3 fault model if it ruptured across more than one of the “spikes” on the red curve.