note on the bias in the estimation of the serial correlation coefficient of ar(1) processes

7/31/2019 Note on the Bias in the Estimation of the Serial Correlation Coefficient of AR(1) Processes

1/11

Statistical Papers manuscript No.(will be inserted by the editor)

Note on the bias in the estimationof the serial correlation coefficientof AR(1) processes

Manfred Mudelsee

Institute of Meteorology, University of Leipzig, Stephanstrae 3,04103 Leipzig, Germany

Received: November 30, 1999; revised version: July 3, 2000

Abstract We derive approximating formulas for the mean andthe variance of an autocorrelation estimator which are of prac-tical use over the entire range of the autocorrelation coefficient. The least-squares estimator

n1i=1 ii+1 /

n1i=1

2i is stud-

ied for a stationary AR(1) process with known mean. We usethe second order Taylor expansion of a ratio, and employ thearithmeticgeometric series instead of replacing partial Cesarosums. In case of the mean we derive Marriott and Popes (1954)formula, with (n

1)1 instead of (n)1, and an additional term

(n 1)2. This new formula produces the expected decline tozero negative bias as approaches unity. In case of the varianceBartletts (1946) formula results, with (n 1)1 instead of (n)1.The theoretical expressions are corroborated with a simulationexperiment. A comparison shows that our formula for the meanis more accurate than the higher-order approximation of White(1961), for || > 0.88 and n 20. In principal, the presentedmethod can be used to derive approximating formulas for otherestimators and processes.

Acknowledgement: The present study was carried out while

the author was Marie Curie Research Fellow (EU-TMR grantERBFMBICT 971919) at the Institute of Mathematics andStatistics, University of Kent, Canterbury, UK.

Statistical Papers Vol. 42, No. 4 (2001), P. 517 to 527

Note: This version has been checked as identical to the

printed version. Manfred Mudelsee, 06 December 2001


2/11

Manfred Mudelsee

1 Introduction

Consider the following well-known least-squares estimator of theserial correlation coefficient of lag 1 from a time series i (i =

1, . . . , n) sampled from a process Ei with known (zero) mean: = n1

i=1

ii+1

n1i=1

2i . (1)

Let Ei be the stationary AR(1) process,E1 N(0, 1), Ei = Ei1 + Ui, i = 2, . . . , n , (2)

with Ui independent and identically distributed as N(0, 1 2) and || < 1. Then the variance and the mean of

are ap-

proximately given by the general formulas (for arbitrary lag) of

Bartlett (1946) and Marriott and Pope (1954), respectively:

var() var()B = (1 2)n

, (3)

E() E()MP = 2 n

. (4)

(The indices refer to the two studies.)An example for an application is fitting an AR(1) model to

real data for which the mean is a priori known and can besubtracted. Formulas for E(

) can be used for bias correction,

whereas var() gives the uncertainty of the estimated fit param-eter .Approximations (3) and (4) are based on the second order

Taylor expansion of a ratio. The primes indicate that the formu-las are further based on the replacement of partial Cesaro sums,like

n1r=n+1

1 |r|

n

|r|

with+

r=

|r|,

see, for example, Priestley (1981, section 5.3). Evidently, thequality of the latter simplification is in doubt when || is large(Bartlett 1946; Kendall 1954). In particular: as 1, all time

518


3/11

Estimation of the serial correlation coefficient

series points become equal and 1. However, (4) fails toproduce that zero bias as 1. This failure is also sharedby published approximations of higher order. For example, theformulas of White (1961) for estimator (1) and process (2) are,

to our best knowledge, the most accurate in terms of powers of(1/n):

var() var()W = 1n 1n2 + 5n3

1

n 9

n2+

53

n3

2

12n3

4, (5)

E(

) E(

)W =

1 2

n+

4

n2 2

n3

+

2

n23

+ 2n2 5. (6)

For = 0, Eq. (6) cannot produce zero bias.The true zero variance of as 1, on the other hand, is

produced by Bartletts formula, (3), whereas Whites formula,(5), fails to do so.

The same weaknesses of formulas (4), (5) and (6) exist alsofor 1.

Since approximations (3)(6) are not intended for large ||,our focus is to derive approximating formulas for var() and E()which can be used over the entire range of , in particular for 1. This is done in Section 2. Thereby we also use the sec-ond order Taylor expansion. However, no further simplificationis made: we employ the arithmeticgeometric series instead ofreplacing partial Cesaro sums. This yields an approximation forE() which consists of Marriott and Popes formula, (4), withan additional term; our approximation produces zero bias for 1. We further confirm Bartletts result, (3), for var().The order of the error for the different approximating formulasis considered. The theoretical expressions are examined over theentire positive range of with a simulation experiment (Section

3), for different values of n. As a practical consequence of thisexperiment, we give the range of for which our formula for E()should be used instead of (6) or (4).

519


4/11

Manfred Mudelsee

2 Series expansions

For the derivations we assume = 0.

2.1 Variance

We write (1) as

= 1n

n1i=1

ii+1

1

n

n1i=1

2i

= c /v , say,

and then we have, up to the second order of deviations from themeans of c and v, the variance

var(

) = var(c/v) var(c)

E2(v) 2 E(c)cov(v, c)

E3(v)

+ E2(c)var(v)E4(v)

. (7)

We now apply a standard result for quadravariate standard Gaus-sian distributions with serial correlations j (e. g. Priestley 1981,p. 325), namely

cov(aa+s, bb+s+t) = ba ba+t + ba+s+t bas,

from which we derive, for i following process (2):

cov

1

n

n1

a=1aa+s,

1

n

n1

b=1bb+s+t

=1

n2

n1a,b=1

(|ba| |ba+t| + |ba+s+t| |bas|), (8)

where the symbol

a,b represents a double summation. Withoutfurther simplification we can use this equation to directly derivethe various constituents of the right-hand side of (7). Firstly, weconsider var(c), which is given by putting s = 1 and t = 0 in Eq.(8):

var(c) = var

1

n

n1

i=1ii+1

=1

n2

n1a,b=1

2|ba| +n1a,b=1

|ba+1| |ba1| .

520


5/11


We find, by summing over the points of the a-b grid in a di-agonal manner, and by using the arithmetic-geometric series,that

n1a,b=1

2|ba| = (n 1) + 2n2i=1

i 2(ni1)

= (n 1)

2

1 2 1

+ 2(2n4 2)

(1 2)2 (9)

and

n1a,b=1

|ba+1| |ba1| = (n 1) 2 + 2n2i=1

i 2(ni1)

= (n

1)2

1 2+ 2

2

+2(2n4 2)

(1 2)2 . (10)

Equations (9) and (10) together give var(c). Further, by puttings = 0 and t = 1 in Eq. (8),

cov(v, c) = cov

1

n

n1i=1

2i ,1

n

n1i=1

ii+1

=2

n2

n1

a,b=1 |ba| |ba+1|,

where

n1a,b=1

|ba| |ba+1| = 2n3 +n2i=1

(2 i + 1) 2(ni)3

= (n 1) 2

1

1 2 1

+ 2(2n5 3)

(1

2)2

+(2n3 1)

(1 2) . (11)

521


6/11

Manfred Mudelsee

Similarly, by putting s = t = 0 in Eq. (8),

var(v) = var

1

n

n1

i=12i

=2

n2

n1a,b=1

2|ba|,

which is given by (9). We finally need

E(v) =n 1

n(12)

and

E(c) =(n 1)

n. (13)

All three terms contributing to the right-hand side of (7) havea part proportional to (n 1)2. However, these parts cancel outto give:

var() var()M = (1 2)(n 1) . (14)

(The index refers to this study.) This expression for var()Mholds also for = 0.

2.2 Mean

We have, up to the second order of deviations from the meansof c and v, the mean

E() = E(c/v) E(c)E(v)

cov(v, c)E2(v)

+ E(c)var(v)

E3(v). (15)

Making use of previous expressions for cov(v, c), var(v), E(v) andE(c), we obtain

E(

) E(

)M = 2

(n

1)+

2

(n

1)2( 2n1)

(1

2). (16)

This expression for E()M holds also for = 0. Letting 1,E()M approaches 1.

522


7/11


2.3 Approximation error

Anderson (1995) showed that the error of approximation (3) isof

O(n2) and the error of (4) is of

O(n1). Since we also used

the second order Taylor expansion, the same orders are validalso for the approximating formulas derived here, (14) and (16)respectively. Thus, our expression for the variance, (14), cannotbe distinguished from Bartletts result, (3). This is interestingsince we have not replaced partial Cesaro sums, while Bartletthas. The first two terms of our expression for the mean, (16),correspond to Marriott and Popes result, (4).

An additional source of approximation error is due to replac-ing partial Cesaro sums; this error, for || large, may be largerthan the error due to finite n.

White (1961) derived formulas (5) and (6) in a different man-

ner. He calculated the kth moment of (c/v) via the joint momentgenerating function m(c, v), with c and v defined as here (Section2.1) without the factors 1/n,

E(k) = 0

vk

. . .

v2

km(c, v)

ck

c=0

dv1 dv2 . . . dvk.

He expanded the integrand to terms of order n3 and 4.

3 Simulation experiment

We generated a set of 250 000 time series from process (2) foreach combination of n (20, 100) and (densely distributed overthe interval [0; 1[). For each time series was calculated, yield-ing sample means, E()S, and sample standard deviations. Thesecan be compared with the theoretical values calculated from for-mulas (14) and (16), respectively. We include Whites (1961)formulas, (5) and (6), in the comparison. Figure 1 shows plotsof the negative bias, E(), and ofvar().

In general, the deviations between the theoretical results andthe simulation results decrease with increasing n, as is to beexpected.

Up to a certain value of , Whites (1961) formulas performbetter than ours since they are more accurate with respect topowers of (1/n). For larger , our formulas give results closer to

523


8/11

Manfred Mudelsee

0.00

0.04

0.08

0 .0 0 .2 0 .4 0 .6 0 .8 1 .0

0.00

0.10

0.20

0 .0 0 .2 0 .4 0 .6 0 .8 1 .0

- E (^)

n = 20

SQRT[va r (^)]

n = 20

Figure 1: Simulation experiment. Top panel: negative bias, bot-

tom panel: standard deviation. Simulation results (dots). Theo-retical result, our formulas, (14) and (16), respectively, (heavyline). Theoretical result, Whites (1961) formulas, (5) and (6),respectively, (light line).

the simulation, particularly the decline to zero negative bias as approaches unity. This decline is caused by the term proportionalto (n1)2 in (16). For large, and n small, this term contributesheavier to E().

Let = be defined by

|E()M E()S| < |E()W E()S| for > =,that means, = gives the range for which our formula for themean, (16), is closer to the simulation than Whites formula,

524


9/11


0.00

0.01

0.02

0.0 0 .2 0 .4 0 .6 0 .8 1 .0

0.00

0.05

0.10

0.0 0 .2 0 .4 0 .6 0 .8 1 .0

- E (^)

n = 100

SQRT[var (^)]

n = 100

Figure 1: (Continued.)

(6). Since = should depend on n, we carried out additionalsimulations as above for n = 10, 50, 200, 500 and 1000 (wherenecessary with an increased number of time series in order tosuppress the simulation noise). It turned out that for n 20,=(n) is given by the intersection between the two theoreticalformulas (E()M = E()W) which is plotted in Fig. 2. As canbe seen in Fig. 2, =(n) does not strongly depend on n. Thepractical consequence is that our formula for the mean, (16),should be used when

|

|is larger than about 0.88 and n

20.

In case of the variance, var()W gives results closer to the sim-ulation than var()M (and var()B) for nearly the entire rangeof (Fig. 1). We consider this is due to the fact that var() de-

525


10/11

Manfred Mudelsee

2 5 2 51 0 1 0 0 10 00

n

0. 80

0. 82

0. 84

0. 86

0. 88

0. 90

=

Figure 2: Comparison between Whites approximation, (6), and

our approximation, (16), for the mean. For > = our formulais more accurate, and vice versa (see text for details).

creases monotonically with ||a behaviour that is also shownby Whites formula. The fact that var()B describes the simula-tion better than var()M for 0, is regarded as spurious.4 Extensions

In the same manner as here, approximations can be obtained for

other estimators (cf. Box et al. 1994, section A7.4) of the serialcorrelation coefficient. Consider, for example, the YuleWalkerestimator YW = n1i=1 ii+1 /ni=1 2i . Because here the sumsdiffer in the number of terms, the analogue to Eq. (8) has an ad-ditional term, and the approximations for the variance and themean are more complicated than Eqs. (14) and (16), respectively.However, the principal behaviours for approaching unity aresimilar in case of YW: the decline of the negative bias (to a valueof 1/n) and the approach of zero variance. An experiment withn = 20 and 250 000 simulations, corroborating the approxima-tions, showed that YW is more biased and has a smaller variancethan the least-squares estimator. For > 0.5, the mean squarederror (MSE, given by bias2 + variance) of YW is larger than theMSE of the least-squares estimator, and vice versa for < 0.5.

526


11/11


Our approximation formulas apply also to AR(1) processeswith constant but unknown variance, and asymptotic stationaryAR(1) processes (transient behaviour). In principal, it should bepossible to obtain formulas for estimators of arbitrary lag in the

same manner as here.

References

Anderson, O.D. (1995). The precision of approximations to mo-ments of sample moments. J. Statist. Comput. Simul. 52, 163165.

Bartlett, M.S. (1946). On the theoretical specification and sam-pling properties of autocorrelated time-series. J. Roy. Statist.Soc. Supplement 8, 2741 (Corrigenda: (1948) J. Roy. Statist.Soc. Ser. B 10, No. 1).

Box, G.E.P., Jenkins, G.M., Reinsel, G.C. (1994). Time seriesanalysis: forecasting and control, 3rd edn. Prentice-Hall, Engle-wood Cliffs, NJ, 598 pp.

Kendall, M.G. (1954). Note on bias in the estimation of auto-correlation. Biometrika 41, 403404.

Marriott, F.H.C., Pope, J.A. (1954). Bias in the estimation ofautocorrelations. Biometrika 41, 390402.

Priestley, M.B. (1981). Spectral Analysis and Time Series, Aca-demic Press, London.

White, J.S. (1961). Asymptotic expansions for the mean andvariance of the serial correlation coefficient. Biometrika 48, 8594.

527

note on the bias in the estimation of the serial correlation coefficient of ar(1) processes

Documents