course peru salas obey all

Riesgo Hidrológico de EventosExtremos en Condiciones

No-Estacionarias

Jose D. Salas

Colorado State University, USA

J. Obeysekera

South Florida Water Management District, USA

Laboratorio Nacional de Hidráulica, Universidad Nacional de Ingeniería, Lima Perú

Lecture Outline• Introduction. Causes of change in extreme events• Review if basic concepts of probability and statistics• Models of extreme events for stationary and non-

stationary conditions• Parameter estimation and model selection for

stationary conditions• Other model alternatives for extreme events• Introduction to software R for modeling extreme

events

Lecture Outline (cont.)• Introduction to software R for modeling extreme

events• Parameter estimation and model selection for non-

stationary conditions• Return period and risk for stationary and non-

stationary conditions• Examples of analysis of extreme events based on

stationary and non-stationary conditions• Hands on experience in using software R for

analyzing extreme events under stationary and non-stationary conditions

Resources

No-Estacionarias

Jose D. Salas

Colorado State University, USA

J. Obeysekera

South Florida Water Management District, USA

Introduction: Causes of changes in extreme events

• Course objectives

• What is expected to be learned from the course

• Summary of the course content

• Format

• Materials and references

• Others

Introduction

Outline

• Extreme hydrologic events

• Types of extreme value data

• Types of changes (non‐stationarity) in hydrologic data

• Causes of non‐stationarity

Introduction

Extreme Hydrologic Events

• Floods

• Low flows and droughts

• Max. and min. precipitation

• Max. and min. temperature (heat waves)

• Maximum wind

• Max. and min. storages (e.g. reservoir, groundwater levels, soil moisture, snow pack, glaciers)

• Max. sea levels

• Max. erosion rates, sediment transport, and sediment deposition

• Maximum water quality pollutant concentrations

• Many others

Drastic damages and changes produced by the extraordinary flood originated from Hurricane Mitch of October 1998. (a) a river and a bridge in Guatemala, Source: Time Magazine, Nov. 1998, and (b) the Choluteca River in Tegucigalpa, Honduras, Source: National Geographic Vol. 196(5), p. 111, Nov. 1999.

Extreme Floods

(a) (b)

Tous Dam at Rio Jucar, Spain breached and caused a flood of about 15,200 m3/s along River Jucar in October of 1982. Source: Dr. J. Marco‐Segura, U. Valencia.

Extreme Floods

Fort Collins flood, July 1997

Extreme Floods

Washed out road over Milford Dam near Milford, Kansas in July 1993(source: cover of ASCE Civil Engineering Magazine, Vol. 64(1), January.)

Mississippi flood of 1993, USA(source unknown)

Extreme Floods

Damage of extreme flood occurred in Venezuela, Dec. 15, 1999(source: Central University in Caracas, Prof. Marco P. Rivero)

Venezuela Venezuela

Extreme Floods

Queensland, Australia, 2011

Pakistan, 2010 India, 2010 China

Extreme Floods

ʺThe frequency (of extreme weather situations) is way up,ʺ Andrew Cuomo, Governor of New York, 10/31/2012

Extreme Droughts

Dust bowl, Midwest US 1930’s(source unknown)

Hayman fire near Cheesman Reservoir(source: Denver Water)

Extreme Droughts

A photo of a wildfire in the west in 2000 due to drought conditions, Aug.

6, 2000 in the Bitterroot Valley National Forest, MT.

Drought in the Amazon RiverSource: G. Chamorro, SENAMHI

Extreme Droughts

Cheesman Reservoir, ColoradoDrought 2002

(source: Denver Water)

Dillon Reservoir, ColoradoDrought 2002

(source: Denver Water)

Types of Extreme Value Data : Block Maxima

Blocks1 2 3 4 5

Block length could be:One YearSeason (wet and dry)

Example: Daily Precipitation

Types of Extreme Value Data : Peaks Over Threshold (POT)

• In hydrology, this is known as Partial Duration Series

Threshold

Definitions of Low Flows

Time series of daily flows, 1951‐2000

Definition of Low Flows (1)

Unit time period Tu

v1 vjvm

The d‐day low flow for a given unit time period Tu is:

Q’ = min(v1, v2, …, vm)

where m = number of d‐day flows in a unit time period,

e.g. m=356 for d=10 and Tu=1 year

For N unit time periods (e.g. for N years of record) we

will get the sequence

Q’1 , Q’2 , . . . , Q’N

Then, frequency analysis must be performed to get the

T‐year d‐day low flow. Note that for low flows

T = 1/q (q=non‐exceedance probability)

Low flow for a variable duration

In this case the duration of low flows is a random variable

A threshold flow Qo is selected and the following low flow

variables can be defined (refer to the following figure)

d* = max(d1, d2, . . . , dm) max. duration of low flows

v* = max(v1, v2, . . . , vm) max. deficit

I* = max(I1, I2, . . . , Im) max. intensity of low flows

Other quantities can be defined such as averages or

maximums, etc.

Series: d*1 , d*2 , . . . , d*N to do frequency analysis.

d1 d2 d3

v1 v2 v3

Example of low flow duration frequency analysis

The daily flows of Edisco River, SC for the period 1951‐2000

Example of low flow duration frequency analysis

Extreme multiyear droughts

Poudre River annual flows at the Mouth of the Canyon (1884‐2002)

Drought definition and properties

Critical droughts can be extracted from the time series

Types of Changes (non‐stationarity)in Hydrologic Data

• Trends (increasing and decreasing)

• Shifts (upward or downward)

• Mixed

• Seasonality (periodicity, cycles)

• Pseudo‐ cycles

• Clustering

• Others

Trends (increasing floods)

(a) Abjerjona Basin (b) Little Sugar Creek Basin.

μt = μ0 ‐ ‐ ‐ μt = μ0 + a t

Trend in the mean and in the standard deviation

Trends (increasing and decreasing)in sea levels

(a) Key West, Florida (b) Adak, Alaska

Example: c = ‐ 1.34 mm/yr, b = 2.71 x 10‐5, e = 1079 mm,σ = 111.3 mm, and ε = ‐ 0.26 for the Adak gage.

Shifts in Hydrologic Data

Niger River annual flows. Example of upwards and downwards shifts

St. Johns River (Florida) Annual FloodsShift related to the AMO

Shifts in Flood Data

Annual Cycle in Hydrologic Data

Monthly streamflow data

Daily Cycle in Hydrologic Data

Hourly rainfall for the month of July for Denver Airport

Causes of Non‐stationarity

• Human intervention in river basins (watersheds)Construction of hydraulic structuresDamsDiversionsTunnelsGroundwater pumpingOthers

Land use changesUrbanizationIrrigation and farming activitiesDeforestationBuilding transportation systemsOthers

Industrial development and operationsMining activitiesOthers

• Human intervention in river basins (watersheds)

• Natural climate variabilityEffects of low frequency (large scale) phenomena such as ENSO (years) PDO (decades), and AMO (multi‐decades)

• Climate change due to increase of green‐house gasesincrease in air temperatureincrease in moisture of the airhydrologic effects uncertain and still debatable

Effect of construction of hydraulic structures along de Han River in Korea (source Dr. D.R. Lee)

Human intervention: Construction of hydraulic structures

Effects of construction of hydraulic structures along de Han River in Korea (source Dr. D.R. Lee)

Human intervention: Construction of hydraulic structures

(a) Aberjona River, WinchesterMassachusetts

(b) Little Sugar Creek, Charlotte North Carolina

Human intervention: effects of urbanization

Natural climate variability: Effects of low frequency

33 years

40 years

26 years 32 years

coldcold

Effect of natural climate variability, PDO(source: Akintug and Rasmussen, 2005)

Effect of natural climate variability, AMOsource: Dr. J. Obeysekera, SFWMD

Natural climate variability: Effects of low frequency

Final Remarks

No-Estacionarias

Jose D. SalasColorado State University, USA

J. ObeysekeraSouth Florida Water Management District, USA

Review of Basic Concepts of Probability, Statistics, and Time Series

Outline

• Random events

• Probability of random events

• Random variable

• Probability laws

• Independent and dependent random variables

• Population moments

• Moments of linear functions of random variables

• Moments of non‐linear functions of random variables

• Central limit theorem

Review of Basic Concepts of Probability

Review of Basic Concepts of Statistics

Outline

• Random sample

• Sample moments

• Moments of sample moments

• Estimation (estimates and estimators)

• Methods of estimation

• Properties of estimators

• Confidence limits on population parameters

• Confidence limits on population quantiles

Review of Basic Concepts of Statistics

Review of Basic Concepts of Time Series

Final Remarks

Riesgo Hidrólogico de EventosExtremos en Condiciones

No-Estacionarias

Models of Extreme Events for Stationary andNon‐Stationary Conditions

Outline

• Typical models for extreme events (stationary conditions)

• General extreme value (GEV) model (stationary conditions)

• General extreme value (GEV) model (non‐stationary conditions)

Models of Extreme Events for Stationary and Non‐Stationary Conditions

• Typical models for extreme events (stationary conditions)

Stationary data

Typical models for extreme events (stationary conditions)

Lognormal

Gamma (Pearson)

Log‐Pearson Type III

General extreme value

the model is called Logistic

Generalized logistic

‐ Exponential if

‐ Uniform if

Generalized Pareto

Vilfredo Pareto1848‐1923

Dr. Paul MielkeProfessor Emeritus, CSU

Mixtures and products

Example: Mixture of three distrib. (Waylen & Caviedes, 1986)

General Extreme Value (GEV) Model(stationary conditions)

• Assume block maxima coming from a series of independent and identically distributed (i.i.d.) observations:

• Distribution of Mn (assuming iids):

• Then for large n, and some an and bn:

G(z) belongs to one of these:

Emil Gumbel1891‐1966

ʺIt seems that the rivers know the theory. It only remains to convince the engineers of the validity of this analysis.ʺ

Maurice Frechet1878‐1973

Waloddi Weibull1887‐1979

Type III(ξ<0)Type I(ξ=0) Type II(ξ>0)

Non‐stationary data

Annual maximum floods

Built‐out?

Non‐stationary data

Mean and maximum sea levels for some sites in the USA

General Extreme Value (GEV) Model(non‐stationary conditions)

Final Remarks

No-Estacionarias

Parameter Estimation and Model Selection forStationary GEV Models

Outline• Parameter estimation and quantile estimation for GEV

modelsMOM, PWM, ML, BayesianProfile Likelihood

• Uncertainty of parameters and quantilesApproximate standard errors (delta method)Better approximation (profile likelihood)

• Model selectionLikelihood ratio (deviance statistic)AICDiagnostic plots

Parameter Estimation & Model Selectionfor Stationary GEV Models

Parameter estimation and quantileestimation for GEV models

Example: Umpqua flood data

PWM estimates

Likelihood ratio test

ML estimates and standard errors

Calculations using R software

Standard errors of parameters

Variance‐covariance matrixEstimation of quantiles and

confidence limits

Estimation of quantiles for T=5, 10, 25, 50, 100, 250, 500 years and theirconfidence limits

*Keep θi constant and maximizeℓ(θ) with respect to all otherparameters, θ‐I

*Repeat for different θi

ℓp(θi)

Profile likelihood for θi

Chi‐square distribution

Confidence intervalfor θi

3.8415/2for k=1

Profile likelihood confidence limits for θi

Confidence Interval

Shape Parameter 25‐yr quantile

Inference using profile likelihood

218 390

‐0.11 0.24

0.0514

Delta method gives:(‐0.24,0.73)

Delta method givesZq:(193.7 341.57)

Results based on profile likelihood

Model selection based on Deviance statistic and AIC

Results for the UmpquaFlood data based on GEV

Final Remarks

No-EstacionariasOther model alternatives for extreme events

Credit:Jana Sillmann

J. Obeysekera (‘Obey’)South Florida Water Management District, USA

Types of Extreme Value Data : Block Maxima (review)

Blocks1 2 3 4 5

Block length could be:One YearSeason (wet and dry)

Models based on r-largest statistics

Daily Rainfall (Fort Lauderdale, Florida)

5 largest values of Rainfall every year

Models based on r-th largest statistics

, … . ,

, … ,

Modeling the r-th Largest Order values

, … , 1,2, . . ,

Likelihood

Example: 10 largest sea-levels in Venice

Year r1 r2 r3 r4 r5 r6 r7 r8 r9 r10

1931 103 99 98 96 94 89 86 85 84 79

1932 78 78 74 73 73 72 71 70 70 69

1933 121 113 106 105 102 89 89 88 86 85139

Example

1 -222.7 111.1(2.6) 17.2 (1.8) -0.077 (0.074)

5 -732.0 118.6(1.6) 13.7(0.8) -0.088 (0.033)

7 916.5 119.1(1.47) 13.25(0.7) -0.09(0.029)

10 1139.1 120.5 (1.36) 12.8 (0.55) -0.113 (0.019)

r-largest order statistics for Venice sea levels

(*ignore trend for the moment)

Global Warming Protesters

Peaks Over Threshold (POT) Models

Motivation: Can we

use all “extreme”

values above a given

threshold?

Threshold Example: Daily Precipitation

Peaks Over Threshold - Theory

Stochastic model:

It can be shown that (Coles,2001):

Generalized Pareto Distribution (GPD)

For large enough u and Y=X-u,

Case: 0 Threshold Selection is important

Can we get the equivalent GEV parameters from GPD?

Need , , from , is the same for both GEV and GPD

Need to estimate , Need some new formulation to estimate the

third parameter

Poisson-GPD Model

The number of N exceedances of the threshold level, u, has a Poisson distribution with mean

Assuming N ≥ 1, the excess values Y1 ,..,Yn

are IIDs from GPD

For z > u, the probability that the annual maximum of Y is,

Poisson-GPD model (cont.)

This will be identical to the form of the GEV if,

Knowing (=# above u/total #) we can now compute the equivalent GEV parameters

A more elegant approach: Point Process Model (PP)

Interpretation of extreme value behavior that unifies all models (GEV, GPD, Poisson-GPD)

Leads to a “likelihood” (to be discussed) that enables a more natural formulation of non-stationarity in threshold excesses than from GPD alone

Poisson Process (PP) Model (preview)

Domain, D=[0,T] x u,∞ X is viewed as two-

dimensional, non-homogenous Poisson process (t, x u

The rate is,

The number of points in A, N(A) is a Poisson Process with mean:

, also

Poisson Process (PP) Model (Cont.)

Advantages: Direct estimation of GEV parameters

is not a function of u (unlike in GPD)

Easier to implement in the “non-stationary” setting (to be discussed later)

Summary of Models

Block Maxima GEV

Gumbel

Others

Peaks Over Threshold (POT) GPD

Poisson-GPD

Generalize Pareto Distribution (GPD) –Maximum Likelihood Estimation

Recall:

Two parameters to estimate:

MLE for GPD - Spreadsheet

=-$K$5*LN($Q$5)-(1+1/$Q$6)*SUM(M7:M364)

File:Fortgpd.xlsx

Poisson Process (PP) Model (review)

Domain, D=[0,T] x u,∞ X is viewed as two-

dimensional, non-homogenous Poisson process (t, x u

The rate is,

The number of points in A, N(A) is a Poisson Process with mean:

, also

Poisson Process (PP) Model –Inference

Advantages: Direct estimation of GEV parameters

is not a function of u (unlike in GPD)

Easier to implement in the “non-stationary” setting (to be discussed later)

Fort Collins Precipitation data again

Point Process (PP) Model - MLE

Likelihood function (Coles 2001, chapter 7)

Ii is a small interval around xi , I is the full range

GEV parameters directly:

MLE for PP - Spreadsheet

=-$O$6*(1+$O$10*($O$7-$O$8)/$O$9)^(-1/$O$10)-$K$8*LN($O$9)-(1+1/$O$10)*SUM(L10:L367)

File:Fortpp.xlsx

Dependent Sequences – Declustering

Average Daily Temperature

Clusters for different r

r=1,2r=3

Declustering Procedure

Use an empirical rule to define clusters of exceedences

Identify maximum excess within each cluster

Assume cluster maxima are independent, and that the conditional excess is distributed as a generalized Pareto

Fit the GPD to the cluster maxima

Change to Return Level Formula

Rate at which clusters must be taken into account. m observation return level:

is the extremal index

Let nu = number of exceedances above u

nc = number of clusters above u

Declustering of Fort Collins Rainfall> load("Fort.RData")

> attach(a)

> dcRain1<-dclust(Rain,u=0.4,r=1)

> names(dcRain1)

[1] "xdat.dc" "ncluster" "clust"

>gpd<-gpd.fit(Rain,0.4,npy=214)

>gdc=gpd.fit(dcRain1$xdat.dc,0.4,npy=214)

gpd$nexc=358 dcRain1$ncluster=288

gpd$mle: 0.394(0.031) 0.164(0.058)

gdc$mle: 0.446(0.037) 0.130(0.060)

Quantile(100-yr)

gpd: 5.05 (3.84,7.89)

gdc: 4.79 (3.71,7.16)

Non-stationary GEV models Parameters are function of time (or any

other variable-”covariates”)

Examples:

Non-stationary GEV: Examples (cont.)

Change Point Models

Math Classic..

Final Remarks

No-EstacionariasIntroduction to modules of the software R for

modeling extreme events

What is R

Most cost effective statistical package (why?)

R is a system for statistical computation and graphics.

Also a programming language, high level graphics, interfaces to other languages

The R language is a dialect of S which was designed in the 1980s

What is R (Cont.)

an effective data handling and storage facility,

a suite of operators for calculations on arrays, in particular matrices,

a large, coherent, integrated collection of intermediate tools for data analysis,

graphical facilities for data analysis and display either on-screen or on hardcopy, and

a well-developed, simple and effective programming language which includes conditionals, loops, user-defined recursive functions and input and output facilities

Installation of R

http://www.r-project.org/

Select Comprehensive R Archive Network (CRAN) mirror site

Download R for Windows (or for the particular operating system)

Install base (install R for the first time)

Provides basic functions which let R function as a language

Base package and contributed packages

For a complete list of functions, use library(help = "base")

Help for R

http://www.r-project.org/

http://www.rseek.org/

http://archive.org/details/TheRBook

Downloading R

Doing more..

Need to get “contributed packages”

http://cran.r-project.org/web/packages/available_packages_by_name.html

> help("install.packages")

Eg. Installing package “extRemes” > install.packages("extRemes")

Will ask for CRAN mirror site

>getwd() to find current working directory

>setwd(“path”) to set the working directory - or use R.gui()

Data structures 1/6 Vector A list of numbers, such as (1,2,3,4,5) R: a<-c(1,2,3,4,5) or a=c(1,2,3,4,5) Command c creates a vector that is assigned to object a

Factor a = c(1,2,2,4,4,5)

A list of levels, either numeric or string R: b<-as.factor(a) Vector a is converted into a factor b:1 2 2 4 4 5

with Levels: 1 2 4 5

Data structures 2/6 Data frame A table where columns can contain numeric and

string values R: d<-data.frame(a, b)

Matrix All columns must contain either numeric or string

values, but these can not be combined R: e<-as.matrix(d) Data frame d is converted into a matrix e

R: f<-as.data.frame(e) Matrix e is converted into a dataframe f

Data structures 3/6

List Contains a list of objects of possibly different types. R: g<-as.list(d) Converts a data frame d into a list g

>g=gev.fit(x) g is now a list object >names(g) check names of the objects in g "trans" "model" "link" "conv" "nllh"

"data" "mle" "cov" "se" "vals"

g$mle gives the values of mle object:301.8081211 169.9874853 0.3544377

Data structures 4/6 Some command need to get, for example, a matrix, and do

not accept a data frame. Data frame would give an error message.

To check the object type: R: class(d)

To check what fields there are in the object: R: d R: str(d)

To check the size of the table/matrix: R: dim(d)

To check the length of a factor of vector: R: length(a)

Data structures 5/6

Some data frame related commands: R: names(d) Reports column names

R: row.names(d) Reports row names

These can also be used for giving the names for the data frame. For example: R: row.names(d)<-c("a","b","c","d","e") Letters from a to e are used as the row names for

data frame d Note the quotes around the string values!

R: row.names(d)

Data structures 5/6 Naming objects:

Never use command names as object names!

If your unsure whether something is a command name, type to the comman line first. If it gives an error message, you’re safe to use it.

Object names can’t start with a number

Never use special characters, such as å, ä, or ö in object names.

Underscore (_) is not usable, use dot (.) instead: Not acceptable: good_data

Better way: good.data

Object names are case sensitive, just like commands

Reading data 1/2

Command for reading in text files is:read.table(”suomi.txt”, header=T, sep=”\t”)

This examples has one command with three arguments: file name (in quotes), header that tells whether columns have titles, and sep that tells that the file is tab-delimited.

Reading data 2/2 It is customary to save the data in an object in

R. dat<-read.table(”suomi.txt”, header=T, sep=”\t”)

Here, the data read from file suomi.txt is saved in an object dat in R memory.

The name of the object is on the left and what is assigned to the object is on the right.

Command read.table( ) creates a data frame.

Using data frames Individual columns in the data frame can be accessed using

one of the following ways:

Use its name: dat$year

dat is the data frame, and year is the header of one of its columns. Dollar sign ($) is an opertaor that accesses that column.

Split the data frame into variables, and use the names directly: attach(dat)

Use subscripts

Subscripts 1/2 Subscripts are given inside square brackets

after the object’s name: dat[,1] Gets the first column from the object dat

dat[,1] Gets the first row from the object dat

dat[1,1] Gets the first row and it’s first column from the

object dat

Note that dat is now an object, not a command!

Subscripts 2/2 Subscripts can be used for, e.g., extracting a subset of the data:

dat[which(dat$year>1900),]

Now, this takes a bit of pondering to work out…

First we have the object dat, and we are accessing a part of it, because it’s name is followed by the square brackets

Then we have one command (which) that makes an evaluation whether the column year in the object dat has a value higher than 1900.

Last the subscript ends with a comma, that tells us that we are accessing rows.

So this command takes all the rows that have a year higher 1900 from the object dat that is a data frame.

Writing tables To write a table: write.table(dat, ”dat.txt”, sep=”\t”)

Here an object dat is written to a file called dat.txt. This file should be tab-delimited (argument sep).

Write.csv(dat, ”dat.csv”, row.names=FALSE)

This file can be opened in Excel

Commonly used R commands in this course

Command What it does

>R start R>?plot Get help on 'plot' command>ls() Check the object space>class(a) Check the data type of object a

>head(a) What the data object looks like (few rows printed)>names(a) names of object a>a=c(1,2,3,4,5) Create a vector object a with values 1:5>c=cbind(a,b) Combine vector objects a and b into the object c

>plot(x,y) plot y versus x

>is.na(a) A logical vector showing which values in a are missing

>sum(is.na(a)) Number of missing values in object a>b=a[‐20,] Remove 20th row of a and place it in b

>a=read.csv("file.csv",header=T)Read data in "file.csv" which has a header and place in to object a

>write.csv(a,"file.csv") Write the object a into csv file called "file.csv"

>a=seq(5,100,5) Create a sequence of numbers, 5 to 100 with increments of 5

Creating a fancy plot

fout= “myplot.png”

#set the device for plotting

png(fout,pointsize=12,units="in",width=8,height=6,res=350)

par(mar=c(5.1,6.1,3.1,3.1))

par(font=2,font.lab=2,font.axis=2,cex=1.5,cex.axis=1.1,cex.lab=1.1)

#now plot

plot(yrs,x,pch=19,xlab="Year",ylab=“Discharge”, col="red")

title(“myplot”)

#set the device off

dev.off()

Reading and Manipulating Data>dat=read.csv(“sample.csv”,header=TRUE)

#notice missing values denoted as “NA”

>class(dat)

>head(dat)

>attach(dat)

>amean=tapply(Rain,Year,mean,na.rm=T)

>momean = tapply(Rain,Month,sum,na.rm=T)

>sum(is.na(dat))

#setting missing values to zero

>dat[is.na(dat[,3]),3]=0

>plot(Rain)

>boxplot(Rain~Month)

Compiling & Running a functionmyplot <- function(Year,Rain,outflag=0) {

if(outflag != 0 ) png("rainplt.png")

amean = tapply(Rain,Year,sum,na.rm=T)

yrs = as.numeric(names(amean))

plot(yrs,amean,pch=19,ylab="Rain",col="red",cex=1.5)if(outflag !=0 ) dev.off()return(list(yrs=yrs,amean=amean))}

>v = myplot(Year,Rain)

> v$yrs[1] 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905> v$amean1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 52.35 62.47 73.28 46.77 40.95 55.96 34.54 37.02 25.51 42.34 52.11

Quitting R

Use command q() or menu choise File->Exit. R asks whether to save workspace image. If you

do, all the object currently in R memory are written to a file .Rdata, and all command will be written a file .Rhistory.

These can be loaded later, and you can continue your work from where you left it.

Loading can be done after starting R using the manu choises File->Load Workspace and File-> Load History.

A powerful IDE for R (www.rstudio.com)

Extremes Package Demo

library(extRemes)

extremes.gui()

Task ismev/extRemes evd evir fExtremes POT VGAM

Data generation

gev gen.gev rgev rgev gevsim, gumbelsim rgev

gpd gen.gpd rgpd rgpd gpdSim rgpd rgpd

GEV fitting

MLE gev.fit fgev gev, gumbel gevFit, gumbelFit gev

Profile Likelihoodgev.prof,gev.profxi,gev.parameterCI profile.evd

Diagnostics gpd.giag

GPD fitting

Threshold mrl.plot mrlplotmeplot,

findthreshmrplot, findThreshold,

mxfPlot mrlplot,lmomplot

gpd.fitrange tcplot tcplot

MLE gpd.fit fpot gpd gpdFit fitgpd gpd

Profile Likelihoodgpd.prof, gpd.profxi, gpd.parameterCI profile.evd gevrlevelPlot

gpd.pfrl,gpd.pfscale, gpd.pfshape

Fisherbasedgpd.firl,gpd.fiscale,

gpd.fishape

bootstrappingboot.matrix, boot.sequence

PP fittingMLE pp.fit fpot pot pointProcess fitpp

decluster declust, decluster deCluster

CovariatesGEV gev.fit fgev

GPD gpd.fit ?

PP pp.fit 195

Final Remarks

No-EstacionariasParameter estimation and model selection for

non‐stationary conditions

Climate change

Extreme Data Types

Stationary Stochastic properties do not change with time

Non-Stationary Stochastic properties vary with time Trends (mean, frequency and intensity)

Cycles (diurnal, annual)

Because of co-variation with other variables (e.g. Flood = f(El Nino phenomenon)

Non-stationary GEV models Parameters are function of time (or any

other variable-”covariates”)

Examples:

Nonstationary GEV Models (cont.)

Change Point Model “Seasons” model

Parameter Estimation (Non-stationary case)

Non-stationary GEV model: Zt = GEV( (t), (t), (t))

Shape parameter is difficult to estimate and it is unrealistic to model it as a smooth a function of time

Log-likelihood:

t , t , t T

Parameter Estimation (Cont.)

Non-stationary Gumbel

Two parameters, but both functions of time

Parameter Estimation (Cont.)

Example 1

Parameters, = ( 0 , 1 , , )

Example 2

Parameters, = ( 0 , 1 , 2 , , )

Approximate Standard errors derived using the same techniques that we discussed for stationary case

Math Classic..

Example: Aberjona River, Winchester, Massachusettes

Typical of basins where land use changes have caused increasing floods

Nonstationarymean, variability?

R and Excel Solutions a=read.csv("Aberjona_1102

500.csv",header=T)

cov=yrs-mean(yrs)

cov=as.matrix(cov,ncol=1)

gev.fit(x,ydat=cov,mul=1)

Output:

$nllh 410.931

$mle 319.3587323 2.8814896 163.3669736 0.3040809

$se 25.3638274 1.0190117 21.1793573 0.1337115

Confidence Limits for parameters: Approximate Normality of MLEs

For large n:

ExpectedInformationMatrix

Response variable: Discharge

[1] "cov.selected = Cov"

Convergence successfull![1] "Convergence successfull!"

[1] "Maximum Likelihood Estimates:“ MLE Stand. Err.

MU: (identity) 319.35873 25.36373

Cov: (identity) 2.88149 1.01901

SIGMA: (identity) 163.36697 21.17922

Xi: (identity) 0.30408 0.13371

[1] "Negative log-likelihood: 410.931027606394"

Parameter covariance:

[,1] [,2] [,3] [,4]

[1,] 643.318930 7.19927261 358.945815 -1.33843641

[2,] 7.199273 1.03838421 4.686030 -0.05198826

[3,] 358.945815 4.68603046 448.559343 -0.37964699

[4,] -1.338436 -0.05198826 -0.379647 0.01787876

extRemes package

Confidence Limits using Bootstrapping

Recall that Z ~ GEV( t , t ,

Then e = [Z- t / tis GEV~(0,1,

The basic approach is to bootstrap the residuals, e and fit multiple models

Fit Z ~ GEV( (t), (t), )

Compute residuals, e

For k = 1, Nsamples Resample residuals with

replacement = enew

Recreate z = Znew using ( (t),(t),

Fit Znew ~ GEV( (t), (t), )

Compute ZT for the desired T

Compute 5th and 95th interval using Nsamples of ZT

Model Selection (Likelihood Ratio Test) Let a model, M1=f(θ(1), θ(2)) and M0=f(θ(1)=0, θ(2)). M0 is a subset of M1. The

question is, Is model M1 any better than M0? Deviance Statistic:

Reject M0 in favor of M1 if

where cα is the (1-α) quantile of the χ2k

distribution dimension of θ(1)

Alternatively:

Model Selection based on AIC & BIC

Akaike’s Information Criteria (AIC) AIC(k) = -2llh (k)+2 k, k=number of parameters

Select the model which has the minimum AIC

Bayesian Information Criterion (BIC) BIC(k) = -2llh(k) + k ln T k = number of parameter

T = sample size

Select the model which has the minimum BIC

Model Diagnostics (Non-stationary) case

~ , , Standardized variable

Note: follows a Gumbel distribution

Model Diagnostics (Cont.)

Propbability plot:

, exp exp 1,2, … ,

Quantile plot:

, log log ; 1,2, … ,

Modeling Non-stationarity -Summary

Fit various modeling using MLE

Select an appropriate model using AIC, BIC and Likelihood Ratio Test as criteria

Compute Return Level, and Risk for a given Return Period

Compute confidence intervals for parameters and return level

Example: Aberjona River, Winchester, Massachusettes

Typical of basins where land use changes have caused increasing floods

Nonstationarymean, variability?

Aberjona River – First Stationary Case

>dat=read.csv("Aberjona_1102500.csv",header=TRUE)

>head(dat) (yrs x)>attach(dat)>gum<- gum.fit(x) #fit Gumbel$gum$mle: {337.4(28.1) 209.3(23.0)}$gum$nllh= 419.7192> gum.diag(gum)>AICgum=2*(gum$nllh+2) =843.4384

>gev<- gev.fit(x) #fit GEVgev$mle {301.8(25.5) 170.0(22.3)

0.354(0.13)} gev$nllh=414.9693>gev.diag(gev)

Modeling Non-staionarity using covariates

g = gev.fit(x, ydat = cov, mul=c(1,..), sigl=c(1,..), shl=c(1,..), siglink = exp, muinit=c(…),…)

Covariate matrix, cov

Year SOI AMO

1945 …. ….

1946 …. ….

R-Demo

Aberjona River - Model Checking & Non-stationary Modeling>D= -2*(gev$nllh-gum$nllh)

> chi=qchisq(0.95,1)

> p=pchisq(D,1,lower.tail=FALSE)

AICgev=2*(gev$nllh+3)

D = 9.499 ; chi =3.84; p=0.002;

AICgev= 835.9386

D > Chi, and AICgev < AICgum

GEV is chosen over Gumbel

#Probability Weighted Moments

> lmr<-lmom.ub(x)

> pwmgev<-pargev(lmr)

#nonstationarity in location parameter

>cov <- as.matrix(yrs - mean(yrs),ncol=1)

>gevmu <- gev.fit(x,ydat=cov,mul=1)

gevmu$nllh=410.931

gevmu$mle {319.4(25.4) 2.88(1.02) 163.4(21.2) 0.304 (0.134)}

> D= -2*(gevmu$nllh-gev$nllh)

> chi=qchisq(0.95,1)

> p=pchisq(D,1,lower.tail=FALSE)

> AICgevmu=2*(gevmu$nllh+4)

D=8.076 chi=3.841459 p=0.004 AICgevmu=829.8621

> Select GEV-MU(t) over GEV!

Aberjona River – Model Comparison

IndexModel Name Parameters nllh Comparison D chi p AIC

1 gum , 419.72 843.4

2 gummu (t), 414.01 2 vs. 1 11.42 3.84 0.001 834.0

3 gumsc , (t) 412.21 3 vs. 2 15.01 3.84 0.000 830.4

4 gummusc (t), (t) 406.23 4 vs. 2 15.56 3.84 0.000 820.5

5 gev , , 414.97 5 vs. 1 9.50 3.84 0.002 835.9

6 gevmu (t), , 410.93 6 vs. 5 8.08 3.84 0.004 829.9

7 gevsc , (t), 411.99 7 vs. 6 5.96 3.84 0.015 832.0

8 gevmusc (t), (t), 405.79 8 vs. 7 10.29 3.84 0.001 821.6

Non-stationary Model –GEV{ t t ,

Confidence Limits using Bootstrapping

Recall that Z ~ GEV( t , t ,

Then e = [Z- t / tis GEV~(0,1,

The basic approach is to bootstrap the residuals, e and fit multiple models

Fit Z ~ GEV( (t), (t), )

Compute residuals, e

For k = 1, Nsamples Resample residuals with

replacement = enew

Recreate z = Znew using ( (t),(t),

Fit Znew ~ GEV( (t), (t), )

Compute ZT for the desired T

Compute 5th and 95th interval using Nsamples of ZT

Snippets of code#initial guess R0library(BB)p0 = 1/T0yp0 = -log(1-p0)R0 = mu0 - (sig/xi)*(1-yp0^(-xi))#compute Return level for the desired value of T – finding root using dfsaned = dfsane(par=R0,fn=myfun,T=T,Nmax=Nmax,mut=mut,sig=sig,xi=xi,quiet=TRUE,control=list(trace=FALSE))Rm = d$parconv <- d$convergence

myfun <- function(x,T,Nmax,mut,sig,xi) {pt = 1- exp(-((1+(xi/sig)*(x-mut))^(-1/xi)))pt[is.nan(pt)]<-1ptc = cumprod(1-pt)px =matrix(1,Nmax,1)for(i in 1:Nmax) {if(i == 1) px[i] = pt[1] else px[i] = ptc[i-1]*pt[i]}t=1:Nmaxex = sum(t*px)y <- T - ex#print(ex)y}

Rest of the code (for boostrapping)for(k in 1:Nsample) {enew <- sample(e,replace=TRUE)znew <- enew * g$vals[,2]+g$vals[,1]gnew<-gev.fit(znew,ydat=cov,mul=1)muslope = gnew$mle[2]mu0=gnew$vals[yrs==lastyr,1]+(consyr-lastyr)*muslopesig = gnew$mle[3]xi = gnew$mle[4]mut = mu0 + (t-1)*muslopeR0new = mu0 - (sig/xi)*(1-yp0^(-xi))

.. Continued to the right

d = dfsane(par=R0new,fn=myfun,T=T0,Nmax=Nmax,mut=mut,sig=sig,xi=xi,quiet=TRUE,control=list(trace=FALSE))Rmnew = d$parcvals[k] <- d$convergenceRms[k]<-Rmnew….}

#5th and 95th confidence limits?r <- quantile(Rms,probs=c(0.05,0.95))

Results - Parameters

Results – Return Level (Design Quantile)

Trend with a change point Little Sugar Creek at Archdale

Drive in Charlotte, North Carolina (Gumbel Distribtion)

>cov = yrs

>cov[yrs <= 1945] <- 1945

cov:1945 1945 1945 1945 1945 1945 1945 1945 1945

1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1945 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975………………

>cov=matrix(cov-mean(cov),ncol=1)

>gum=gum.fit(x)

>gumu=gum.fit(x,ydat=cov,mul=1)

Two Change points Mercer Creek,

Washington State

>yr1 = 1970

>yr2 = 1986

>cov=yrs

>cov[yrs <= yr1] <- yr1

>cov[yrs >= yr2] <- yr2cov: 1970 1970 1970 1970 1970 1970

1970 1970 1970 1970 1970 1971 1972 1973 1975 1977 1978 1980 1981 1982 1983 1985 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986 1986

>gev = gev.fit(x)

>gevmu=gev.fit(x,ydat,mul=1)

Sea Level Rise – Modeling extremesKey West tide gauge>a=read.csv("KeyWest.csv",header=TRUE)> head(a)Year Mean Max1913 1495.4 20421914 1481.2 2070>attach(a)>gevmu=gev.fit(Max,mul=1,ydat=as.matrix(Year-mean(Year)))>g=lm(Mean~Year) #regression

Non-linear change possible in the future Relation shop mean and max! Can we use that property? – later

Offset=552 mm

Sea Level Rise Projections (in US)

Modeling with Covariatesdata(fremantle)

Year SeaLevel SOI

1897 1.58 -0.67

1898 1.71 0.57

…attach(fremantle)ydat=cbind(Year-mean(Year),SOI-mean(SOI))gev=gev.fit(SeaLevel)gevmu=gev.fit(SeaLevel,ydat=ydat,mul=1)gevmusoi =gev.fit(SeaLevel,ydat=ydat, mul=c(1,2))

So which model is better?

D<- -2*(gevmu$nllh-gev$nllh)

chi<-qchisq(0.95,1)

p<-pchisq(D,1,lower.tail=FALSE)

AIC=2*(gevmu$nllh+4)

Fremental Extreme Sea Levels –Model Comparison

IndexModel Name Parameters nllh Compare D chi p AIC

1 gev , , -43.567 NA NA NA NA -81.13

2 gevmu (t), , -49.914 2 vs. 1 12.69 3.84 0.0004 -91.83

3 gevmusoi (t,soi), , -53.898 3 vs. 2 7.97 3.84 0.0047 -97.79

3 gevmusoi (t,soi), , -53.898 3 vs. 1 20.66 5.99 ~0 -97.79

gevmu is better than gev

Adding SOI appears to improve the model further (compare 3 vs. 2)

Atlantic Multi-Decadal Oscillation (AMO) index as a covariate St. Johns River, Florida. Floods

influenced by the phase of AMO

>dat=read.csv(“Stjohn.csv”,header=T)

>attach(dat)

Year StjQ

1944 3300

1945 9230

>ydat=rep(1,1,length(Year))

>ydat[yrs <= 1969] = -1

>ydat<-matrix(ydat,ncol=1)

>gev=gev.fit(StjQ)

>gevmu=gev.fit(StjQ,ydat=ydat,mul=1)

gev$nllh=400.7574

gevmu$nllh=396.4719

D=8.571; Chi=3.841; p=0.003

AICgev=403.8; AICgevmu=400.5

Math Classic..

Final Remarks

No-Estacionarias

Return Period and Risk for Stationary andNon‐stationary Extreme Conditions

Return Period and Risk Under Stationary Conditions

Return Period Under StationaryConditions

Design flood and constant values of exceeding (p) and non‐exceeding (q = 1‐p) probabilities throughout years 1 to t,

i.e. stationary condition (Salas and Obeysekera, 2013)

Distribution of the “Waiting Time”for Stationary Conditions

Distribution of the Waiting Time(also known as “First Arrival Time”)

Flood occurrenceFlood occurrence

Return Period Under StationaryConditions

Hydrologic Risk Under StationaryConditions

Return Period and Risk Under Non‐Stationary Conditions

Fig.5 Example of non‐stationary annual flood data

Developments to deal with Non‐Stationarity

Return Period and Risk Under Non‐Stationary Conditions

Distribution of the “Waiting Time”for Non‐Stationary Conditions

Return Period for Non‐Stationary Conditions

Hydrologic Risk for Non‐Stationary Conditions

Hypothetical Example for DeterminingT and R for Non‐stationary Conditions

Final Remarks

No-Estacionarias

Examples of Analysis of Extreme Events forNon‐stationary Conditions

Examples of Analysis of Extreme Events for Non‐stationary Conditions

Outline

• Examples based on the exponential distribution

• Examples of increasing floods

• Examples of increasing and decreasing sea levels

• Examples of shifting flood regimes

Example Using an Exponential Distribution

Examples of Analysis of Extreme Events for Non‐stationary Conditions

Examples Using the GEV forIncreasing Flood Events

Aberjona River Basin, Winchester, Massachusetts (Vogel et al, 2011)

Aberjona River Basin, Winchester, Massachusetts

Examples Using the GEV forIncreasing and Decreasing Sea Levels

(a) (b)

Further Remarks

course peru salas obey all

salascolorado

usalaboratorio

compute 5thand

likelihood

natural climate

homogenous

nsamples resample

poisson process

Documents

obey gravity update

descendientes de francisco salas

calculus-salas - test bank

franklin salas martinez

obey your body

diseño de salas digestive_endoscopy

paul salas

hold fast: obey

salas manhãs

jesus salas

salas araceli action research.pptx

invexo imobiliária · exclusive offices salas a413 2205m*...

catalogo obey / obey catalog

salas febrero 2014

trust & obey

catalogo salas

cruise drft06rr lima, peru – easter island, chile survey...

beatriz antunez, adrian salas, jesus vazquez, adrian salas

santiago (de las salas) salas

cv marina lopez salas