robust scale and autocovariance estimation

Robust scale and autocovariance estimation

Garth Tarr, Samuel Muller and Neville Weber

School of Mathematics and Statistics

THE UNIVERSITY OF SYDNEY

EMS 2013

Scale Autocovariance Conclusion

Outline

Robust Scale Estimator, Pn

Covariance

Autocovariance

Long RangeDependence

Short RangeDependence

Inverse CovarianceMatrix Estimation

Covariance Matrix

Outline

Robust Scale Estimator, Pn

Covariance

Autocovariance

Long RangeDependence

Short RangeDependence

Inverse CovarianceMatrix Estimation

Covariance Matrix

Outline

The robust scale estimator Pn

Autocovariance estimation using Pn

Conclusion and key references

History of scale efficiencies at the Gaussian (n = 20)R

1931 1974

Pairwise mean scale estimator: Pn

• Consider the U -statistic, based on the pairwise mean kernel,

Un(X) :=

)−1∑i<j

Xi +Xj

• Let H(t) = P ((Xi +Xj)/2 ≤ t) be the cdf of the kernelswith corresponding empirical distribution function,

Hn(t) :=

)−1∑i<j

I{Xi +Xj

2≤ t}, for t ∈ R.

Definition (Interquartile range of pairwise means)

Pn = c[H−1n (0.75)−H−1

n (0.25)],

where c ≈ 1.048 is a correction factor to ensure Pn is consistentfor the standard deviation when the underlying observations areGaussian.

Influence curve

• Hampel (1974) defines the influence curve for a functional Tat distribution F as

IC(x;T, F ) = limε↓0

T ((1− ε)F + εδx)− T (F )

where δx has all its mass at x.• Serfling (1984) outlines the IC for GL-statistics.

Influence curve for Pn (Tarr, Muller and Weber, 2012)

Assuming that F has derivative f > 0 on [F−1(ε), F−1(1− ε)] forall ε > 0,

IC(x;P , F ) = c

[0.75− F (2H−1

F (0.75)− x)∫f(2H−1

F (0.75)− x)f(x) dx

−0.25− F (2H−1

F (0.25)− x)∫f(2H−1

F (0.25)− x)f(x) dx

Influence curve

• Hampel (1974) defines the influence curve for a functional Tat distribution F as

IC(x;T, F ) = limε↓0

T ((1− ε)F + εδx)− T (F )

where δx has all its mass at x.• Serfling (1984) outlines the IC for GL-statistics.

Influence curve for Pn (Tarr, Muller and Weber, 2012)

Assuming that F has derivative f > 0 on [F−1(ε), F−1(1− ε)] forall ε > 0,

IC(x;P , F ) = c

[0.75− F (2H−1

F (0.75)− x)∫f(2H−1

F (0.75)− x)f(x) dx

−0.25− F (2H−1

F (0.25)− x)∫f(2H−1

F (0.25)− x)f(x) dx

Influence curves when F = Φ

−4 −2 0 2 4

Asymptotic variance and relative efficiency

• Tarr, Muller and Weber (2012) show that when the underlyingobservations are independent, Pn is asymptotically normalwith variance, V , given by the expected square of theinfluence function.

• When the underlying data are independent Gaussian,

V (Pn,Φ) =

∫IC(x;P ,Φ)2 d Φ(x) = 0.579.

• This equates to an asymptotic efficiency of 0.86 as comparedwith 0.82 for Qn and 0.37 for the MAD.

! But how does it compare at heavier tailed distributions?

V (Pn,Φ) =

∫IC(x;P ,Φ)2 d Φ(x) = 0.579.

V (Pn,Φ) =

∫IC(x;P ,Φ)2 d Φ(x) = 0.579.

Asymptotic relative efficiency when f = tν for ν ∈ [1, 10]

1 2 3 54 76 98 10

Degrees of freedom

veeffi

1 2 3 54 76 98 10

Degrees of freedom

veeffi

1 2 3 54 76 98 10

Degrees of freedom

veeffi

Asymptotic distribution under short range dependence

Assumption 1: Gaussian SRD

Let {Xi}i≥1 be a stationary mean-zero Gaussian process withautocovariances γ(h) = E(X1Xh+1) satisfying

∑h≥1 |γ(h)| <∞.

Result

Under Assumption 1, it can be shown that,

√n(Pn − σ) =

cΦ√n

n∑i=1

IC(Xi, P ,Φ) + op(1),

hence applying a limit theorem from Arcones (1994) we have√n(Pn − σ)

D−→ N (0, σ2),

where σ =√γ(0) and

σ2 = E(IC2(X1, P ,Φ)

∑k≥1

E (IC(X1, P ,Φ)IC(Xk+1, P ,Φ)) .

Asymptotic distribution under long range dependence

Assumption 2: Gaussian LRD

Let {Xi}i≥1 be a stationary mean-zero Gaussian process withautocovariance sequence γ(h) = E(X1Xh+1) satisfyingγ(h) = h−DL(h), 0 < D < 1, where L is slowly varying at infinity.

Result

Under Assumption 2,

1. If D > 1/2,√n(Pn − σ)

D−→ N (0, σ2),

σ2 = E IC2(X1, P ,Φ) + 2∑k≥1

E IC(X1, P ,Φ)IC(Xk+1, P ,Φ).

!Note that this is the same as the SRD result, however theproof is somewhat more involved.

Assumption 2: Gaussian LRD

Let {Xi}i≥1 be a stationary mean-zero Gaussian process withautocovariance sequence γ(h) = E(X1Xh+1) satisfyingγ(h) = h−DL(h), 0 < D < 1, where L is slowly varying at infinity.

Conjecture

Under Assumption 2,

2. If D < 1/2,

k(D)nDL(n)−1(Pn − σ)D−→ σ

(Z2,D(1)− Z2

1,D(1)),

where k(D) = B((1−D)/2, D) and B denotes the betafunction Z1,D is the standard fBm process and Z2,D is theRosenblatt process.

!This is consistent with results for other scale estimators,e.g. SD and Qn (Levy-Leduc et. al., 2011) and the IQR...

Interquartile range result

Consider equivalent result for the interquartile range,

Tn(x) = cΦ

(F−1n (3/4)− F−1

n (1/4)).

Result

Under Assumption 2 with D < 1/2, Tn satisfies the following limittheorem as n→∞:

k(D)nDL(n)−1(Tn − σ)D−→ σ

(Z2,D(1)− Z2

1,D(1)).

• In establishing this result we used Wu’s (2005) Bahadurrepresentation for sample quantiles under LRD:

F−1n (p)− r =

p− Fn(r)

φ(r)+X2n

φ′(r)

φ(r)+O(nh(D)L1(n)).

!It appears to be non-trivial to extend this result to the pairwisemean empirical distribution function.

Tn(x) = cΦ

(F−1n (3/4)− F−1

n (1/4)).

Result

(Z2,D(1)− Z2

1,D(1)).

F−1n (p)− r =

p− Fn(r)

φ(r)+X2n

φ′(r)

Tn(x) = cΦ

(F−1n (3/4)− F−1

n (1/4)).

Result

(Z2,D(1)− Z2

1,D(1)).

F−1n (p)− r =

p− Fn(r)

φ(r)+X2n

φ′(r)

Empirical densities ARFIMA(0, 0.1, 0); D = 0.8; n = 5000

0.95 1.00 1.05

Scale estimate

87%SD Relative Efficiency

Qn 83%IQR 39%MAD 39%

0.95 1.00 1.05

Scale estimate

Pn 87%SD Relative Efficiency

0.95 1.00 1.05

Scale estimate

Qn 83%

IQR 39%MAD 39%

0.95 1.00 1.05

Scale estimate

Qn 83%IQR 39%

MAD 39%

0.95 1.00 1.05

Scale estimate

1.2 1.4 1.6 1.8 2.0

Scale estimate

1.2 1.4 1.6 1.8 2.0

Scale estimate

1.2 1.4 1.6 1.8 2.0

Scale estimate

Qn 94%

IQR 82%MAD 83%

1.2 1.4 1.6 1.8 2.0

Scale estimate

Qn 94%IQR 82%

MAD 83%

1.2 1.4 1.6 1.8 2.0

Scale estimate

Empirical densities with 2% contamination at 5

1.2 1.4 1.6 1.8 2.0

Scale estimate

QnIQRMAD

1.2 1.4 1.6 1.8 2.0

Scale estimate

QnIQRMAD

1.2 1.4 1.6 1.8 2.0

Scale estimate

QnIQRMAD

1.2 1.4 1.6 1.8 2.0

Scale estimate

QnIQRMAD

Outline

From scale to autocovariance

• Gnanadesikan and Kettenring (1972) highlighted an identitythat relates scale and covariance.

• In the autocovariance setting,

γ(h) = cov(X1, Xh+1) =1

4[var(X1+Xh+1)− var(X1−Xh+1)] .

• For a series of n observations, Xn = {Xt}1≤t≤n,

γP (h) =1

[P 2n−h(X1:n−h+Xh+1:n)− P 2

n−h(X1:n−h−Xh+1:n)].

where X1:n−h are the first n− h observations in Xn andXh+1:n are the last n− h observations.

• The same technique can be used to turn other scaleestimators into covariance and correlation estimators, e.g. Maand Genton (2000) study γQ based on Qn.

Robustness properties

• Following Genton and Ma (1999), we have shown thatinfluence curve, IC((x, y), γP ,Φ), and therefore the grosserror sensitivity of γP can be derived from the IC of Pn.

x−4 −2 0 2 4

IC(x,y)

−3−2−10123

IC((x,y),Pn,Φ)

! IC is bounded and hence the gross error sensitivity is finite.

x−4 −2 0 2 4

IC(x,y)

−3−2−10123

IC((x,y),Pn,Φ)

x−4 −2 0 2 4

IC(x,y)

−3−2−10123

IC((x,y),Pn,Φ)

Breakdown value

• Ma and Genton (2000) show that the breakdown value forautocovariance estimators is (roughly) half that of thecorresponding covariance estimator.

• Consider n = 13

, autocovariance at lag h = 2

• Working with X1:11 ±X3:13

• 4 contaminated observations, denoted by

• Leaves only 3 uncontaminated pairs, denoted by

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X9 X10 X11 X12 X13

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X1 X2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X11 X12 X13

! γP (h) has roughly half the 13.4% breakdown value of Pn.

Breakdown value

• Consider n = 13, autocovariance at lag h = 2

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X9 X10 X11

X12 X13

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X1 X2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X11 X12 X13

Breakdown value

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11

X9 X10 X11

X12 X13

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X11 X12 X13

Breakdown value

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X9 X10 X11 X12 X13

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X11 X12 X13

Breakdown value

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X9 X10 X11 X12 X13

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X11 X12 X13

Asymptotic distribution under short range dependence

Result

If {Xi}i≥1 satisfy Assumption 1 (SRD) then the sequences{Xi +Xi+h}i≥1 and {Xi −Xi+h}i≥1 also satisfy Assumption 1,hence we can use the previous results and apply the functionaldelta method to show,

√n(γP (h)− γ(h))

D−→ N (0, σ2(h)),

σ2(h) = E[IC2((X1, X1+h), γP ,Φ)

∑k≥1

E [IC((X1, X1+h), γP ,Φ) IC((Xk+1, Xk+1+h), γP ,Φ)] .

Result

Under Assumption 2 (LRD),

1. D > 1/2 (same limiting distribution as SRD):

√n(γP (h)− γ(h))

D−→ N (0, σ2(h)).

2. D < 1/2 (based on conjectured result):

k(D)nD

L(n)(γP (h)−γ(h))

D−→ γ(0) + γ(h)

2(Z2,D(1)− Z2

1,D(1)),

where k(D) = Beta((1−D)/2, D), Z1,D is the standard fBmprocess, Z2,D is the Rosenblatt process and

L(n) = 2L(n)+L(n+h)(1+h/n)−D+L(n−h)(1−h/n)−D.

Empirical densities γ•(1) ARFIMA(0, 0.1, 0); D = 0.8; n = 5000

0.95 1.00 1.05 1.10First order autocovariance estimate

87%SD Relative Eff.

Pn 87%SD Relative Eff.

Qn 83%

IQR 39%MAD 39%

Qn 83%IQR 39%

MAD 39%

1.5 2.0 2.5 3.0 3.5 4.0 4.5First order autocovariance estimate

Qn 94%

IQR 82%MAD 83%

Qn 94%IQR 82%

MAD 83%

Empirical densities γ•(1) with 2% contamination at 5

QnIQRMAD

Outline

Summary

1. Aim

• Efficient and robust scale and autocovariance estimation.

2. Method

• Pn scale estimator transformed using the GK identity.

3. Results

• 86% asymptotic efficiency at the Gaussian and highasymptotic efficiency at heavier tailed distributions.

• Robustness properties transfer from Pn to γP .

• In certain LRD settings robust estimators have very highefficiencies relative to the standard deviation.

Summary

1. Aim

2. Method

3. Results

Summary

1. Aim

2. Method

3. Results

References

Genton, M. and Ma, Y. (1999).

Robustness properties of dispersion estimators.

Statistics & Probability Letters, 44(4):343–350.

Gnanadesikan, R. and Kettenring J. R. (1972).

Robust estimates, residuals and outlier detection with multiresponse data

Biometrics, 28(1):81–124.

Hampel, F. (1974).

The influence curve and its role in robust estimation.

Journal of the American Statistical Association, 69(346):383–393.

Levy-Leduc, C., Boistard, H., Moulines, E., Taqqu, M.S. and Reisen, V.A.(2011).

Robust estimation of the scale and of the autocovariance function ofGaussian short- and long-range dependent processes

Journal of Time Series Analysis, 32(2):135–156.

References

Ma, Y. and Genton, M. (2000).

Highly robust estimation of the autocovariance function

Journal of Time Series Analysis, 21(6):663–684.

Serfling, R. J. (1984).

Generalized L-, M -, and R-statistics.

The Annals of Statistics, 12(1):76–86.

Tarr, G., Muller, S. and Weber, N.C., (2012).

A robust scale estimator based on pairwise means.

Journal of Nonparametric Statistics, 24(1):187–199.

Wu, W.B., (2005).

On the Bahadur representation of sample quantiles for dependentsequences.

Annals of Statistics, 33(4):1934–1963.

robust scale and autocovariance estimation

Documents

robust dynamic state estimation in power...

censoring-robust estimation in observational...

multivariate robust estimation of dcc-garch

locally robust semiparametric estimation · locally robust...

a robust nonparametric estimation framework for implicit...

robust estimation for differential equations, time … ·...

doubly robust and locally efficient estimation …

robust estimation, random matrix theory and applications ......

two-phase kernel estimation for robust motion...

robust semiparametric estimation...robust semiparametric...

universiti putra malaysia robust estimation...

robust estimation in parameter learning

non-parametric approximations for anisotropy estimation in...

robust methods and model selection€¦ · robust scale...

robust kernel density estimation

robust estimation in pulse fluorometry astudy …

distributed robust power system state estimation

robust estimation

robust regression - robust estimation of regression...

robust estimation of correlation coefficients among soil...