lecture 9: nonparametric modelling€¦ · lecture 9, page 9 • polynomial • running mean •...

Lecture 9, page 1

Lecture 9: Nonparametric modelling What is the problem of parametric models? • We have no reason to believe the assumptions (e.g., linearity,

normality, stationarity etc.) • We have no reason to chose a specific model In reality: • We discover insufficiencies of our parametric models Nonparametric models may be used for • Visualisation of data • exploring functional relationships • identify candidate models

___________________________________________

C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 1 of 1

Lecture 9, page 2

Illustration: Abundance of herring (This first example: not time series model. Illustration of principles)

Herring length in mm

30 40 50 60

010

030

050

0

Abu

ndan

ce o

f her

ring

(10^

9)

-> Obviously not linear trend -> Not clear how to fit data

___________________________________________


Lecture 9, page 3

Another example (inspired by Howell Tong):

Kilpisjärvi data

Year

Trap

inde

x

1960 1970 1980 1990

05

1015

2025

30

Reversed data

Year

Trap

inde

x

1960 1970 1980 1990

05

1015

2025

30

The tops are not symmetric. The build-up phase is not equal to the crash phase => Non-linearities If we know the biological relationship, we may model this parametrically, i.e., y = α1x + α2x2+ε What if we don't?

___________________________________________


Lecture 9, page 4

Model formulations

Changin

Classical linear model: [ ] ppp XXXXYE βββ +++= ......| 1101

Generalised Linear Model (GLM) [ ]( ) ppp XXXXYEg βββ +++= ......| 1101

Additive Model: [ ] )(...)(...| 1101 ppp XfXfXXYE +++= β

Generalised Additive Model (GAM): [ ]( ) )(...)(...| 1101 ppp XfXfXXYEg +++= β

g to well known notation…

___________________________________________


Lecture 9, page 5

-> Diffe(generadistribugamma)

Classical linear model: [ ] ptptpttt XXXXXE −−−− +++= ααα ......| 1101

Generalised Linear Model (GLM) [ ]( )

pptptpttt XXXXXEg −−−− +++= ααα ......| 1101

Additive Model: [ ] )(...)(...| 1101 ptptpttt XfXfXXXE −−−− +++= α

Generalised Additive Model (GAM): [ ]( ) )(...)(...| 1101 ptptpttt XfXfXXXEg −−−− +++= α

rent link functions can be used to make the model additive lise the model); logit, reciprocal, log, etc. This depends on what tion the data have (Normal, binomial, gamma, poisson, inverse

___________________________________________


Lecture 9, page 6

-> Often it is sufficient to stay at the “GLM”, i.e., we log the data and achieve an additive model

Kilpisjärvi data

1960 1970 1980 1990

01

23

Reversed data

1960 1970 1980 1990

01

23

Example:

)loglog(1

2211 −− −−−= tt NN

tt eNN αα )log( tt Nx =

22111 −−− −−= tttt xxxx αα

2211)1( −− −−= ttt xxx αα

___________________________________________


Lecture 9, page 7

Nonparametric smoothers The function that transforms the data is called a smoother.

Herring length in mm

30 40 50 60

010

030

050

0

Abu

ndan

ce o

f her

ring

(10^

9)

Our smoother need some properties: It should be smooth (mathematical: Twice differenctiable) It should summarise the data; on the expense of fit: What we do is then to penalise the number of parameters (in figure: If we use all the data points, we can fit the data perfectly). Mathematical expression:

( ) ∑=

=t

jjj ySxs

100

S0 is a smoother (of some complicated form).

___________________________________________


Lecture 9, page 8

Different smoothers

___________________________________________


Lecture 9, page 9

• polynomial

• running mean

• running line

• loess

• gaussian kernel

• smoothing spline: Used a lot in biology – picewise linear regressions that are twice differentiable in knots.

• regression spline

• natural spline

30 40 50 60

010

030

050

0

Is this important? Probably not very important. Not big difference between smoothers.

___________________________________________


Lecture 9, page 10

Possible strategy Preliminarily: • Checking parametric models Is

Xt = f(Xt-1)+εt (some nonlinear shape) significantly better than Xt = α+βXt-1+εt ? (lagged linear regression model) P-values: f1 f2 f3 0.433 0.213 0.77 0.444 0.101 0.723 0.358 0.056 0.656 0.251 0.046 0.55 0.146 0.041 0.375 0.067 0.035 0.17 0.035 0.029 0.081

2.4 2.6 2.8 3.0 3.2

-0.6

-0.2

0.0

0.2

0.4

0.6

2 3 4 5 6 7

-0.6

-0.2

0.0

0.2

0.4

0.6

-4 -2 0 2 4

-0.6

-0.2

0.0

0.2

0.4

0.6

degrees of freedom

p-va

lue

3 4 5 6 7 8 9

0.0

0.1

0.2

0.3

0.4

H0: f1 = linear

degrees of freedom

p-va

lue

3 4 5 6 7 8 9

0.0

0.05

0.10

0.15

0.20

H0: f2 = linear

degrees of freedom

p-va

lue

3 4 5 6 7 8 9

0.0

0.2

0.4

0.6

0.8

H0: f3 = linear

___________________________________________


Lecture 9, page 11

(Software: function "gam" in Splus + +) We want to model, i.e., Xt = f(Xt-1) Stochastic model: Some variability in Xt for a fixed value of Xt-1. i.) Decide on the link-function -> Decided by theory, shape of data etc. ii.) Decide on smoother -> We have seen that the exact form of the smoother is not very important. iii.) Decide on amount of smoothing iv.) Do model selection

___________________________________________


Lecture 9, page 12

iii.) Decide on amount of smoothing Idea to penalise "un-smoothness" of the smoother:

{ } { }∫∑ +−=

b

a

t

jij dttfxfy 2

1

2 )('')( λ

Complexity Model fit

This is done by evaluating the second derivative (which measures how often the data turns). From the formula: The size of the λ-parameter decides on the balance between fit and smoothness.

(When using splines in Splus, the amount of smoothing is determined by the degrees of freedom, d.f, with default=4). Can be tested with, e.g., Cross Validation (CV) or AIC: AIC = –2 maximized log likelihood + 2 # parameters -> But we have no parameters, or no likelihood! AIC = –deviance + 2df(λ) dispersion parameter Both AIC and CV are functions of λ, and can be calculated for as many values of λ as wanted. The λ giving lowest CV or AIC value is selected as the most appropriate model.

___________________________________________


Lecture 9, page 13

iv.) Model selection (One aspect is to "select" the amount of smoothing). Nested models can be tested formally:

[ ]( )[ ]( ) 22112

2111

)4,(...|

)3,()4,(...|

−−−−

−−−−

+==

=+==

ttpttt

ttpttt

XdfXsXXXEgdfXsdfXsXXXEg

β

(The "s()" means smoothing spline) The deviance between the two models is approximately χ2-distributed.

___________________________________________


Lecture 9, page 14

Example: Blowflies – Finding the non-linear shape (Moe et al. 2001) Regard the relationship between larval density and survival from larvae to pupae. Non-parametric approach: No assumptions about the shape of the relationship.

The alternative hypothesis: A linear relationship, i.e., a straight line

20 50 150 400 1000

-2.0

-1.0

0.0

0.5

1.0

Larval density L

f(L)

Formulate a biologically based (parametric) model of the same relationship:

)exp( 114

11

−−+ = t

bt

t

t cLaLLP

20 50 150 400 1000

0.0

0.2

0.4

0.6

0.8

1.0

Larval density L

exp(

a +

f(L))

___________________________________________


Lecture 9, page 15

Example: Fish species richness – Common structure? (Lekve et al. 2008) Regard the relationship between species richness at time t and t-1 and the influence of environmental stochasticity. Non-parametric approach: No assumptions about the shape of the relationship. Model of species richness, S, in fjord f at time t : log[(St+1)/(St-1+1)] = g1f(log(St-1+1)) + g2f(temperaturet)

yt,f = g1f(log(St-1+1)) + g2f(temperaturet)

where g1f is the function of richness-dependent regulation, and g2f is the functions of environmental influence on species richness (here: temperature). Testing for common structure in, say, the function g1f between fjords i and j will be to test

H0: g1,i = g1,j versus

H1: g1,i ≠ g1,j using software developed by Ole Chr. Lingjærde:

___________________________________________


Lecture 9, page 16

1.5 2.0 2.5 3.0 3.5

-1.0

0.0

0.5

1.0

GI

Richness-dependencet

2 3 4 5 6 7

-1.0

0.0

0.5

1.0TJ

GI

Temperaturet

___________________________________________


Lecture 9, page 17

___________________________________________


Assessment Nonparametric approach can be valuable in: • Detecting (and testing for) nonlinearities • Testing for common structure • Finding parametric form of models

lecture 9: nonparametric modelling€¦ · lecture 9, page 9 • polynomial • running mean •...

Documents