THE LEAST SQUARES SPECTRUM, ITS INVERSE TRANSFORM AND AUTQCORRELATION FUNCTION: THEORY AND S O m

APPLICATIONS IN GEODESY

Michael Ruthven Craymer

A thesis subrniûed in conformity with the requirements for the degree of Doctor of Philosophy, Graduate Depattment of Civil Engineering, University of Toronto

@ Copyright by Michad Ruthven Craymer 1998

National Library Bibliothèque nationale du Canada

Acquisitions and Acquisitions et Bibliogaphic Services services bibliographiques

395 W d i Street 395, nie Weuitn* OFEawaON K t A W ûttawaON K I A W Canada Canada

The author has granted a non- exclusive Licence allowing the National Library of Canada to reproduce, loaq distribute or sell copies of this thesis in microform, paper or electronic formats.

The author retains ownership of the copyright in this thesis. Neither the thesis nor substantial extracts fiom it may be printed or otherwise reproduced without the author's permission.

L'auteur a accordé une licence non exclusive pemettant à la Bibliothèque nationale du Canada de reproduire, prêter, districbuer ou vendre des copies de cette thèse sous la forme de microfiche/nlm, de reproduction sur papier ou sur format électronique.

L'auteur conserve la propriété du droit d'auteur qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation,

THE LEAST SQUARES SPECïRUM, ITS INVERSE TRANSFORM AND

AUTOCORRELATION FUNCTION:

THEORY AND SOME APPLICATIONS IN GEODESY

Doctor of Phiiosophy 1998

Michael Ruthven Craymer

Deparmient of Civil Engineering, University of Toronto

To redize the fuil potential of increasingly more accurate measurements,

scientists are now faced with the ta& of modelling ever smaller effects on their

observations to improve their resuits. The problem, however, is that there is often little

understanding of the cause and effect relation between these so-caiied systematic effects

and the measurements. Spectra and autocorrelation functions can be used to help

diagnose and improve the modelling of these systematic effects in measurements.

However, standard techniques for computing spectra and autocorrelation functions

requixe the data to be evenly spaced, which is often not satisfied in practice.

The approach taken here is to develop a general technique for determining

autocorrelation functions for data which are unevenly spaced. This is an indirect method

whereby the systematic effects, represented by the residuals nom an incomplete a priori

deterministic model, are transformed into a power specrrum and then into an

autocorrelation function. To accommodate unevenly spaced data, a general least squares

transfomi and its inverse are developed. The inverse transfonn is used to obtain the

autocorrelation function from the least squares spemrn onginally developed by Vaniik

[1971]. This formulation can accommodate unequaily spaced data, random observation

emrs, arbitrary fî-equency selec tion, arbi trarily weigh ted and coxrelated observations, as

well as the presence of any a priori detemiinistic modeL The conventionai Fourier

transform and spectrum are shown to be just special cases of this more general least

squares formulation. It is also shown how the individual spectrai components in the least

squares spectmm and inverse transfomi can be estimated either independently of or

simuitaneously with each other.

The advanrages and limitations of the least squares transforms and spectra are

illustrated through tests with simuiated data. The technique of using autocorrelation

funceions to mode1 systematic effects is also iiiustrated with two r d applications; one

based on the precise measurement of the extension of a baseline spanning the San

Andreas fault in California, and another based on the measurernent of ellipsoidal heigha

using a GPS receiver under the infiuence of the effects of Selective Availability. These

tests show that the use of hlly populaied weight maaices generally results in an increase

in the value of the standard deviations of the estimated model parameters, thereby

providing more realistic estimates of the uncednties. On the other hand the effect of

comlations among the observations on the least squares estimates of model parameters

was found not to be very significant.

ACKNOWLEDGMENTS

Ta Maty , Sarah, Lisa and Scu~u(eI.

This work is dedicated to my f d y , Mary. Sarah, Lisa and Samuel. It simply

would not have been possible without their saaifice and unfailing support and

understanding for so many years. I owe them a huge debt of gratitude.

I am also deepIy indebted to my supenrisor, Rofessor Petr Vaniîek for a i i his

guidance, advice, generosity and persevering suppoh His tircless and meticdous efforts

in reviewing my manuscripts and patience in dealing with my stubbonuiess are! gratefully

appreciated. He is truiy the quintessentiai supervisor. I codd not have been more

fortunate to have him as my mentor.

1 also thank the members of rny Examination Committee, especiaiiy my Intemal

Appraiser, Rofessor Ferko Csiilag (Geography), and my Extemal Examiner, Rofessor

Douglas E. Smylie ( E h and Atmospheric Science, York University). Their constructive

comments and recornrnendations, together with those £hm the other members of my

Examination Comrnittee, are greatly appreciated

The GPS data used in my tests were kindy provided by William Rescott of the

U.S. Geologicai Survey in Menlo Park. 1 especiaily thank John Langbein, also of the

U.S. Geological Survey in Menlo Park, for supplying the EDM &ta and for generously

taking the t h e to discuss some of the results of my analyses.

1 express my sincere gratitude to my employer, the Geodetic S m e y Division of

Geomatics Canada, and in particular Nomian Beck and Lloyd Nabe for giving me time and

support to complete this work.

Portions of this research were also hinded by various Na@ Sciences and

Engineering Research Council of Canada Operating Grants held by Rof. Petr VanRSek

during the years 1986 to 1990.

Finally, 1 thank the Department of Civil Engineering for giving me the opportunity

to finish rny dissertation after so many years. 1 especiaüy thank Professor Robert Gunn for

his help in this regard

TABLE OF CONTENTS

.. Abstract ............................................................ ,. ,. ............................. A Acknowledgments ............................................................................... iv List of Tables ..................................................................................... k

List of Figures ..................................................................................... x

Chapter 1 . Introduction .................................................................. 1

Chap ter 2 . Basic Concepts of Stochastic Rocesses ................................. 6 2.1 Types of Processes ................................................................ -6

2.2 Determïnistic and Random Processes ........................................... -7 2.3 S tationarity and Ergodicity ........ ..... ....... ... ............................. 7

2.4 Statistical Moments ............................................................... 10

2.5 Covariance and Correlation Functions ......................................... 1 1

2.6 Decomposition of the Observable ............................................... 15

Chapter 3 . The Fourier Transform and Spectmm ............... .. ........... 17 ..................................................... 3.1 FourierSeIiesandIntegrals 17

3 . 2 Fourier Transform ................................................................ 21

3.3 Fourier Spectmm ................................................................. 25

3.4 Convolu tion and Correlation .................................................... 33

3.5 Fast Fourier Transfomi ......................................................... -36

3.6 OtherTransforms ................................................................. 38

............................................ . Chapter 4 The Least Squares Transform 40

........................................................................ 4.1 Introduction 40

............................................. 4.2 MatrixFormofFourierTransfonn 41

........................................................ 4.3 Least Squares Transform 46

........................................... 4.4 Weighted Least Squares Transform -49

................................................... 4.5 EfTm of Deteminhic Mode1 53

...................................................... 4.6 Vector Space Interpretation 57

4.7 Applications ...................................................................... -61

. Chapter 5 The Leas t Squares Spectrum .............................................. 63

........................................................................ 5.1 Introduction 63

5.2 Manix Fom of Fourier Spectrum .............................................. 64

............. 5.3 Least Squares S pectrum ... ...................................... 6 5

................... ..................... 5.4 Weighted Least Squares Spectrum .. 67

5.5 Effect of Deterministic Mode1 ................................................... 71

.................................................................... 5.6 StatisticalTests 74

5.7 Estimation Algorithms ............................................................ 79

.... Chapter 6 . Stochastic Modeiiing of Observation Errors ............... .. 81

6.1 Introduction ........................................................................ 81

..................... 6.2 Direct Autocovariance Function Estimation ............ .,, 82

....................... 6.3 Autocovariance Function Estimation via the Spectrum 83

............................. 6.4 Iteratively Reweighted Least Squares Estimation 84

. Chapter 7 Numerical Tests ............................................................... 86

........................................................................ 7.1 Introduction 86

7.2 Effect of Random Observation Errurs ......................................... -87

........................... Chapter 8 . Some Applications in Geodes y .... ... -125 8.1 Introduction ....... ... ...... -., .................................................... -125

8.2 EDM Deformation Measurements ............................................. -126

8.3 GPS Point Positioning .......................................................... 142

Chapter 9 . Conclusions and Recommendations .................................. 156

References ........................................................................................ 159

LIST OF TABLES

..................... Table8.1 Leastsquaresestunatesoflinear~dmd&tumoffs~ 129

Table 8.2 Least squares estimates of lineâr trend and datum offsets, including

additional d a m ofkt (#5a). ................................................... 134

Table 8.3 Least squares estimates of linear trend and dahm offsets, includùig

additional offset (ma) and usuig estimated N1 observation covariance

matrix based on cornputexi ACF.. ............................................... .140

Table 8.4 Su- of estimated linear trends with and without extra offset and

correlations .......................................................................... 141

Table 8.5 Unweighted and weighted hourly means and their standard deviations

................. (Std) of GPS height measurements over a 24 hou. period.. .148

..... Table 8.6 Twenty of the largest peaks in least squares spectrum in Figure 8.16. .152

Table 8.7 Weighted hourly means of GPS height measurements and their

standard deviations (Sui) over a 24 hour period using correlations

h m ACF based on 24 hours of data ............................................ 153

LIST OF FIGURES

Figure 2.1:

Figure 3.1:

Figure 4.1:

Figure 6.1 :

Figure 7.1

Figure 7 2

Figure 7 3

Figure 7.4

Figure 7 5

A single random process consisting of an ensemble of 4 sample

records (A, B , C, D) ............................................................. -8

Autocomlation functions (ACF) and power spectral density functions

............................................ (SDF) for some special functions.. -27

Commutative diagram for the direct and inverse least squares transfomi,

..... where F denotes the direct transform and F-1 the inverse transform -60

..................... Iteratively reweighted Ieast squares estimation process -85

Periodic tirne series of 100 e q d y spaced points and period 10

(fiequency 0.1 hz) with no observation emrs and with normaily

distributeci randorn errors (standard deviations 1/3 and 2/3) ................. 89

Least squares spectra of time series of 1 0 equdy spaced points and

p e n d 10 (hquency 0 .l) with no observation emrs and with normally

................ disûibuted random emors (standard deviations 1/3 and 2/3) .90

Direct estimation of unbiased autocor~e1ation functions of tirne series of

100 equaily spaced points and pend 10 (fhquency 0.1) with no

observation errors and with nomally dis~buted random errors (standard

deviations l/3 and 2M) .......................................................... .9 1

Comparison of direct and indirect (via LS spectrum) estimation of biased

autoconelation functions of time series of 100 e q d y spaced points and

........................ penod 10 (frequency 0.1) with no observation errors 92

Cornparison of direct and indirect (via LS spectrum) estimation of biased

autoforrelation functions of time series of 100 equaily spaced points and

p e n d 10 (fÎequency 0.1) with mdom obsemation errors (standand

.................................................................... deviation 1/3). -93

Figure 7.6 Cornparison of direct and indirect (via LS qxxtmm) estimation of biased

autocorrelation functions of the series of 100 equaiiy spaced points and

@od 10 (fhquency 0.1) with random observation amrs (standard

...........*....... ............................................... deviation 2/3). ,. -94

Figure 7.7 Periodic time senes of 100 equdy spaced points with period 10

(fkquency 0.1) and comlated random observation enors (standard

deviation 2/3). .................................................................... .97

Figure 7.8 Unweighted and weighted LS spectra (both independent and simuluu~ous

estimation) for periodic time &es of 100 e q d y spaced points with period

10 (fkquency O. 1) and comlated randorn observation mrs (standard

............*.........*..... ............................... deviation 2/3) ., ..... ... .98

Figure 7.9 Direct and unweighted indirect (via unweighted inverse ûansform of

unweighted LS spectrum) estimates of biased autocorrelation fimction

for periodic time saies of 100 equally spaced points with period 10

(frequency O. 1) and conelated random observation emrs (standard

.................................................................... deviation 2/3). -99

Figure 7.10 Weighted indirect estimates of biased autocorreIation function via

weighted inverse LS msform of both independent and simuitaneoudy

estimated LS spectra for peridc time senes of 100 equaily spaced points

with period 10 (frequency 0.1) and correlated random observation emrs

........................................................ (standard deviation 2/3) .100

Figure 7.1 1 Direct and unweighted indirect (via unweighted inverse transfm of

unweighted LS spectrum) estirnates of biased autocorrelation function

for tirne series of 100 equally spaced points with cme1ated random

obsewation m r s only (standard deviation 2/3). ........................... .lot

Fisue 7.12 Weighted indirect estimates of biased autmmlation function via

weightcd inverse LS transfomi of both independent and simultaneously

estMated LS specaa for time series of 100 equally spaced points widi

..... correlated mndom observation errors ody (standard deviation 2/3). -102

Figure 7.13 Periodic t h e series of different lengths of randornly spaced points

(unifody distibuted) with period 10 (muency 0.1) and no random

observation errors.. ............................................................. .IO5 Figure 7.14a LS spectra (independently estimated fr-equency mmponents) up to

different maximum 6requencies for periodic data series of unequally

spaced points with period 10 (kquency 0.1) and no random

observation errors ................................................................ 106

Figure 7. I4b LS spectra (independently estirnateci hquency components) for

different lengths of periodic data series of unequally spaced points

..... with period 10 (fkquency O. 1) and no random observation enors.. .IO7

Figure 7.15 Indirect estimates (via unweighted inverse LS nansform of unweighted

LS spectrum) of biased autocorrelation functions for different lengths of

periodic data series of unequally spaced points with period 10

...................... (frequency 0.1) and no randorn observation enors.. .IO8

Figure 7.16 Direct estirnates (via interval averaging) of biased autocorrelation

functions for different lengths of periodic data series of unequally

spaced points with period 10 (frequency 0.1) and no random

observation errors. ................ ., ............................................. 1 0

Figure 7.17 LS spectra for different sets of simultaneously estimateci fkquencies

for periodic data series of 100 unequally spaced points with period 10

......................... (frequency 0.1) and no random observation errors -113

Figure 7.1 8 Indirectiy estimated LS autocoirelation fûnctions via the LS spectrum

using different sets of simultaneously estimated wuencies f a

periodic data series of 100 unequally spaced points with period 10

Figure 7.19

Figure 7.20

Figure 7.21

Figure 7.22

Figure 7.23

Figure 7.24

Figure 7.25

Figure 8.1

Figure 8.2

(frequency O. 1) and no random observation emns.. ........................ 1 14

Periodic thne series of randomly spaced points with fiequencies 0.1

and 0.25 25 and no random observation ermrs (top), and independent

...................................... estimation of the LS specfiim (bottom). 115

Indirectly estimated ACF via the inverse LS transform of the

independent LS specmim using aU frequencies (top) and of the

sirnultaneous LS spectrum using only the two signifiant specaal

........................................... peaks at 0.1 and 0.25 hz (bottum). .116

Quadratic trend time series with periodic component (kquency

0.01 hz) and no random emm (top); LS spectrum of residuals h m

quadratic trend mode1 (miiddle); LS specuum accoun~g for effects

................................................... of quadratic mode1 (bottom) . I l 8

Evedy sampled 100 point random walk time s& (standard

deviation 1) (top) and its conesponding LS specmun ............ ...... 121

Direct (top) and indirect (bottom) autocorrelation functions for 100

............................................... point random walk data series.. .122

Unevenly sampled 100 point random walk tirne series (top) and its

corresponding LS spectrum ................................................... .123

Indirect estimate of autocorrelation via the independently estimated

LS spectrum for the unevenly sarnpled 100 point random w a k time

............................................................................. series -124

Location of the Pearblossom network in California used to

measure crusta1 deformation with a two-cofour EDM instrument

and location of the HolcombLepage baseline spanning the San

Andreas fault nuuiing through this network [after Langbein and

Johnson, 1997, Figure 11 ....................................................... 128

Changes in length of HolcombLepage baseline. Different

Figure 8.3

Figure 8.4

Figure 8.5

Figure 8.6

Figure 8.7

Figure 8.8

Figure 8.9

Figure 8.10

Figure 8.1 1

Figure 8.12

obsenration groups are denoted by different symbol colomhype

combinations ..................................................................... 128

Cornparison of residuai baseline length changes after rem0va.i of

estimated distance offsets for each o h a t i o n group and a cornmon

linear trend. DBerent observation groups are denotecl by different

symbol colour&p combinations. ............................................ -130

Histograms of lengths of point mplea (''Nyquist pniods'")

comsponding to possible Nyquia fkequencies. Botrom plot gives

................... a more detailed histogram at 1 &y.. ................... ... 13 1

Weighted least squares spectra (independently estimated) of

...... baseline length residuals fiam the deterministic model in Table 8.1 .132

Changes in length of Holcomb to Lepage baseline with additionai

................. datum offset in observation group fkom 1984 to mid- 1992 133

Cornparison of residual baseline length changes after ~emoval of

estimated datum offsets, including additional offset, for each

............ observation group and a comrnon hear trend for ail groups.. ,134

Weighted lest squares spcctra of baseline length residuals h m

................... the detemiinistic mode1 with additional distance offset.. -135

Semi-log (top) and log (bottom) plots of weighted Ieast squares

spectra of baseiine length residuals h m the detemiinistic mode1

with additional datum offset ................................................... .137

Indirect ACF, and enlargement at short lags. estimated h m

zero-padded time series of Holcomb-Lepage length changes with

additional d a t u . offset.. .......................................-............... .139

Variations in deived horizontal (top) and vertical (bottom) GPS

..... ........................... positions over 24 hours at station Chabot .. 144

Variations in recordecl horizontal (top) and vertical (bottom) GPS

................................. positions for the f k t hour at station Chabot. 145

Figure 8.13 hdependently eshmated least squares specaum of 8 s height

........................ measurcments for the first hour (data zempadded) -147

Figure 8-14 Indirect estimate of the biased autocorrelation function Ma the

inverse least squares transform of the least squares specaum for

the first hour.. ................................................................... ,147

Figure 8.15 Unweighted (top) and weighted (bottom) hourly means of GPS

height measiirements over a 24 hou pend.. ............................... -149

Figure 8.16 Least squares specaum (independently esémated) for entire 24

hour data set ..................................................................... -1 5 1

Figure 8.17 Autocorrelation function for en tire 24 hour dam set ........................ -1 52

Figure 8.18 Weighted (top) hourly means of GPS height measurements over

a 24 hour period using correlations obtained h m ACF based on

24 hours of data, and difference with equaüy weighted means without

........................ .................................. correlations (bottom) ,., 154

Chapter 1 Introduction

Recent advances in technology have produced exmmely precise and accurate

meesuring systems that are affected by even the srnallest effects that were once much too

s m d to be noticed. In the past these eEects were considerd to be mndom noise to be

average6 out To realize the fidl potential of their measurements, scientists are now faced

with the ta& of modelling these s d effects in order to improve their predictions. The

problem, however, is that there is ofien little understanding of the cause and effect relation

between these so-cailed systematic effects and the measured observables.

There are basicaily two approaches to d d b u i g or modelling the measured

observations. Detarnuiistic models are used to explicitly describe the behaviour of the

observations in temis of a mathematical mode1 of the physical process. These detemiinistic

models con& of constants and parameters to be estimami. Ofien, however, there is littie

understanding of the physical processes underiying the behaviour of the measurements. In

the other approach, stochastic models mat the measurements, or what remains afier

removing a detemiinistic part, as unpreciictable random (Le., stochastic) quantities.

Stochastic models describe the dependencies between the data and the incomplete

deterministic model in terms of mathematical correlations. These correlations c m be

represented by Nters, polynomiais, correlation functions and spectral density functions.

Because detaministic modehg is usually the preferred approach. correlations are ofien

used to help diagnose and improve the d e t d t i c modeL In cases where this is not

possible, the correlations, if carefully consmcted, can be used to help describe the residuai

systematic effects within the deterministic model.

The least squares estimation technique is primarily used f a fitting deterministic

models to the measurements. However, it is also able to accommodace stochastic modeis

diIough the use of a N l y populated covariance ma& for the observations. There are

different meuiods of deteminhg the variances and covariances that f m the observation

covariance ma&. The most direct method involves determinhg an autocovariance

function that describes the behaviour of various systematic effecrs. The problem with this,

and the main motivation for tbis work, is t h t traditional techniques for cornputing

autocorrelation functions require the data to be evenly spaced. This may not be the case,

especialiy when looking for correlations with some (physicaily meaningful) parameters

given as numerical functions. In practice such functions are often known only for values

of the argument that are unevenly spaced

The usual way of handling unevenly spaced data is to interpolate or approximate the

original series to get an evedy spaced one. Because this approach tends to mode1 the lower

frequency content in the data, the low fkquency behaviour of the measmemenits must be

known. Moreover, the high fkequency components can be lost by the srnoothing effcct of

the in teplation or approximation.

The approach taken here is to develop a more general technique for detemiining

autocorrelation functions for data which are unevenly spaced with respect to quantities

describing the systematic effects. As will be seen later, there are two basic approaches to

estimating autocorrelation functions. The most direct is to compute the autocorrelation

function directly h m the data. In this case, however, there is no satisfactory method of

handüng unevenly spaced points. There are methods based on averaging over larger,

evenly spaced intervals or bins, but using these results in a loss of resolution.

The aitemative approach is to estimate the autocorrelation function indiRctly by fmt

representing the systematic effects in tenns of a power spectrum and then transfomiing this

into an autocorrelation function. This is the approach taken here. Again the problem is that

most techniques for cornputing the power spectrum require evenly spaced &ta as do those

for trandomiirig the power spectrum to the autocorrelation function. The aim hcrc is to

find a more general technique that does not quire evenly spaced data To this en& a

general least squares aansform is developed.

Other methods are also available for determining the variances and covariances of

the observations. The most popular of these are the methods of analysis of variance and

variance-covariance component esDimatia The "analysis of variance" (ANOVA) mthod

(also called factor analysis in statistics) can be found in most standard texts on statistics.

Geodetic applications of the technique are desmibed in detaü by Keiiy [1991] and in a

series of articles by Wassef [1959; 1974; 19761. Essentially the aim of the method is to

divide the measuremena into separate groups (factors which connibute to the overall

variation in the data) and to estimate the variance components for each. The niff;culty in

applying the method is defining a scheme of dividing the observations into separate groups

which characterize some behaviour of the systematic effect king modeiled. m n , the

factors describing the systematic effect c m o t be so discretely defined, rather they are often

of a CO~MUOUS nature that precludes lumping them together into separate and distinct

groups.

Varianceîovariance component estimation, on the other hanci, is based on

modelling detamuiistically the residual variation in the measurements. The variances and

covariances are expressed in tems of linear models relating these components to various

factors describing the systematic effsct. The coefficients (variance and covariance

components) in the variance-covariance mode1 are estimateci together with the parameters in

a least squares solution. The technique is described in detail in Rao and Kleffe [1988] and

has ken applied :O many geodetic problems ( s e , e.g., Grafarend et al. [1980], Grafarend

[1984], Chen et al. [1990]). It can be shown that the analysis of variance method is just a

special case of this more generai approach [Chnanowski et al., 19941 The problem with

applying the mediod is ihat the estimation of the variance-covariance mode1 coefficients

usually needs to be iterated which can result in biased estimates of the variances and

covariances mao and Kleffe, 19881. This can lead to negative variances, which is

unaccep table.

The approach taken here is to model any residual systematic effects remaining after

removing a determinisec model, using autocorrelation functions derived from a power

spectcai density function of the residuals This idea was first proposai by VanîZek and

Craymer [1983a; 1983b] and furtherdeveloped by Craymer [1984]. To accommodate

uneveniy spaced data, the least squares spectrum. developed by VarilEek [1969a], was used

and converted to an autocorrelation fundon using the inverse Fourier transfonn.

However, the inverse Fourier transfomi is not completely compatible with the more general

lest squares specmim. Consequently, a more general leas squares transform and ia

inverse are developed here which are completely compatible with the lest squares specmim

and cm provide coma autocorrelation functions for data that are unevedy spaced

Although applied only to geodetic problems here, this technique should have wide

application in many areas of science where one needs to model or analyze measured data.

Before desaibing the technique, a review of the basic concepts of stochastic

processes and the conventional Fourier transform and spectrum are given. This is followed

by the development of a new "least squares" transform and its inverse, and the

reformulation of the least squares spectrum, originally developed by VaniZek [1969a;

19711, in tems of this new transform. It is then shown how an autocorrelation function

can be derived h m the least squares specmm using the inverse least squares transform,

and how this can be used in a procedure for stochasticaily modeiiing residual systematic

effects. These developmenu are followed by tests with simulateù data to examine

numerically sûme of the limitations of the technique. It is also applied to a couple of

examples in geodesy; the modelling of residual systematic effects in elecmnic distance

measurement (EDM) data and point positionhg data from the Global Positioning System

(GPS). FinaUy, conclusions and recommendations for M e r investigations are given.

Throughout the sequel the following notation is used:

variables/observables italic

vectors lower case, boldface letters

matrice dope rat^^^ upper c m bolciface l e m

functions upper or lower case le-, no boldface

Chapter 2 Basic Concepts of Stochastic Processes

2.1 Types of Processes

A process can be considered to be any kind of physical phewmenon that varies in

some way. We examine such processes by talang measurements of them; Le., by

describhg their physicai behaviour in tams of numencd quantities that cm then be

andysed mathematicdy. These processes are most commody represented as series of

measurements (obse~ations), often raken with respect to t h e (tirne series) or space (spatial

processes). When regardeci more generally as series with respect to any other argument,

these processes are referred to here as simply data series.

Rocesses Nt) are usudy thought of as one-dimensional; that is, varyhg with

respect to a one dimensional argument (t) such as time. However, a process may also be

multidimensional; Le., a function Nt) of a vector of arguments (t) - e-g., processes Nt)

which are functions of three-dimensiond position (x) in space or four-dimensional position

in space-the. One may ais0 encounter multiple processes Nt) of multiple arguments (t).

Rocesses can be classified as either CO~MUOUS or discrete. Examples of

CO~MUOUS processes are the cmstal motions of land masses due to tectonic deformations or

the motions of satellites in orbit about the Earth. On the other han& the accumulated errors

fiom point to point in a geodetic netwmk would be classined as a dimete pmess (in

space). Generaily, one is only able to obtain discrete samples of continuous processes,

primarily due to the na= of data acquisition systems.

2.2 Deterministic and Random Processes

Processes can also be classifiai as d e t e d s t i c and random. What is random?

"Everything and nothing" according to I k 11983, pp. 4054MJ. There is no one test for

detemiining whether a m e s s is either random or detaniinistic. The definitions most

often used are ody subjective or heuristic and a mana of philosophical debate. One person

may consida a process to be randorn noise to be fïitered out while another may consider

the same random process to be a deteministic signal to be modelled.

The most straightfbrward definition is that deterministic irnplies predictability while

random i m p k unpredîctability. Thus what is considered deterministic and what is

considered random depends on what one wishes to mode1 . The deteminïstic part is what

is king predicted or estimated exactiy while the random or stochastic part is that which one

can only predict or estimate with some degree of uncertainty. In the last century,

instruments had rather Liniited precision and much of the variability in a process was

considered to be random or stochastic, so that one could only predict with a great deal of

uncertainty. More recently, however, new measuring techniques have becorne more

precise so that it is now possible to attempt to mode1 ever smaller variations in the data in an

effort to improve the prediction power of the deterministic modei.

2.3 Stationarity and Egodicity

Different realizations of a random process will, in general, not be identical. A

single realization of a process is calleci a sample record. The collection or ensemble of al1

sample records is cdied a random or stochastic pmcess (see Figure 2.1). In a random

process, di sample records are different while in a deterministic pmcess ali samples are

identical.

Figure 2.1: A single random process consisting of an ensemble of 4 sample records (A,

B, C, D). There are 100 values of the argument ranging nom 1 to 100. Ensemble or

sample averages are taken over the four different sample records for each value (e.g., t or

t+@ of the argument; i.e., there are 100 sample averages. Argument averages are taken

over the arguments for each sample record; i.e., there are 4 argument averages.

Random or stochastic processes can be classified as king either stationary or non-

stationary. A pnness is stationary if the statistical propaàes of the process, defied over

the ensemble, are independent of the argument(s) (usuaily time or space). That is, the

statistical moments over alî realizations (e.g., ensemble or sample averages) are the same

for all values of the argument. A non-stationary process is one for which this property is

not sabsfied. Such processes require special techniques to mode1 dieir behaviour (see,

e.g., Bendat and Piersol [IW 11 and Priestley 1198 11).

In practice, different degrees of stationarity exist If the complete statistical

description of the process (Le., all possible statistical moments) is independent of the

argument, the process is said to be completely stationary. If only the f h t few moments are

independent of the argument, the process is considered to be weakly stationary. Rocesses

with a Gaussian probability distribution are completely describeci by only the fint two

moments. In this case, stationarity in only the first two moments infers complete

stationarity . Stationarity can be further classifieci on the basis of ergodicity. A process is

ergodic if the statistical properties taken over the argument (e.g., tirne averages) are

identical to the statistical properties taken over different realizations (e.g., ensemble or

sample averages). The assumption of ergodicity ailows for a considerable reduction in the

number of observations and computations required to determine the statistical pmperties of

a random process. For the sake of simplicity, convenience and, most importantly, costs,

most random processes are assumed to be ergodic in practice, even though there may be

evidence to the contrary.

When dealing with multidimensional (i.e., mdti-argument) spatial processes Nx)

whose arguments (x) define Location and orientation in space, stationarity is ofien

considered in remis of homogeneity and isotxupy. A process is homogeneous if it is

invariant with respect to its location in space and isotropie if it is invariant with respect to its

orientation [Grafarend, 19761.

Throughout this work a i l processes are assumed to be stationary and ergodic. Any

nonstationarity and nonergdcity is assumed to be explicitly modelled detaministically and

is assumcd to disappear when the mode1 is selected properly.

2.4 Statistical Moments

The properties of a random processes can be described by the statistical moments of

their probability density functions. For a single CO~MUOUS random process (or variable)

Nt), at a parti& argument t (hereafter d e d t h e for convenience), the kth-order moment

is given by

where Ep] is the mathemasical expectation operator and P(&t)) is the probabiüty ciensity

function of the randorn variable at time t. The integration is performed over all samp1e

records at time t. This implies that $(t)C E (--,-) n s t be integrable.

Generally, only the f h t two moments are useful in practice. The firt moment or

mean value is the simplest and most conmion measure of a random process. It provides a

measure of the cenaal tendency in the data series. For random processes with discrete

sample records &(t) , i= 1 ,.. .,n, the mean Mt) at argument t is defined by

where n is the total number of sample recoràs (infinie in the limit).

The second-order moment is a measme of the variation in the random process and

is defined by

The second-order central moment is a measure of the variation about the mean and is also

cded the variance @t)? The discrete fomi of the variance can be written as

2.5 Covariance and Correlation Functions

Covariance and correlation functions are generic tem for the more general second-

order moments which provide a measure of the linear dependence between observations at

different values of the argument t. Autocovariance and autocorrelation functions represent

the linear dependence within a single random process. Cross-covariance and cross-

correlation functions represent the linear dependence between a pair of different random

processes.

The autocovariance function C(t,t') is deked by

When the times are the same (i-e., t=t'), qn. (2.5) reduces to the variance ait)? The

cross-covariance function CO^ between two random processes Nt) and Ht) is defined

simihrly as

The autococorrelation function R(t,t') is defked as the normalized autocovariance

function; Le.,

SimilarIyl the cross-correlation function R d t , t 8 ) is the normalized autocorrelation

function:

The autocomlation function is limited to the range

for all t and t'. When the times t and t' are the same, the autocorrelation bction is equal to

one. The same holds for the crossi=orrelation function.

If the mdom pmcess is stationary, the moments are independent of the value of the

argument, 'Zhus, in the above definitions, the expressions are dependent oniy on the time

difference or lag. F-ZX The moments then Rduce to the foIlow fomis:

Simila- expressions to eqns. (2.1 2 ) and (2.12) can be wrinen for the crosscovariance and

cross-correlation functions.

The following wo propemes of these functions are consequences of the

assumption of stationari ty :

1 . The auto/cross-covariance and auto/cross-correlation functions are even functions

of 2; Le.,

2. At lag d the autocovariance hinction is positive and the autocorrelation function is

equal to one; Le..

Ergodicity is probably the most imponant and 0fte.n used assumption in practical

data analysis applications, even when the process is known to be non-ergodic or even non-

stationary. This is done to simpliSr the data acquisition and hanmg procedures.

Stochastic processes are ergodic if their sample moments (e.g., mean, autocovariance, etc.)

can be det&ed from averaging over the argument (e.g., tirne) instead of averaging over

the sample records (see Fi- 2.1). For the meaa and autacovariance function,

The discret- f m of these expressions (for discrete random processes) are given by:

. n-k

where lag q = k AI and Az is the sampling interval. This expression gives an unbiased

estimte of the autocovariance function. Although unbiasedness is desirable, this function

is not positive definite. Constructing covariance matrices h m this leads to singuiar

maaices. It also exhibits so-caiied "wild" behaviour at large lags. For these reasons, the

biased estimate is recommended by Bendat and Piersol[197 1, pp. 3 12-3141 and Priestley

[1981, pp. 323-3241. whae the denominator n-k in eqn. (2.20) is replaced by the constant

n. This resdts in a iùnction that tapers off as the Iag increases. An example of this is

given in the numerical simulations in Chapter 7.

Sirniiar expressions can also be written for the cross-covariance functions. Note

that the inregrations and summations are wormed over the argument t rather than over the

sample records; i.e., unda the assumption of ergodicity the moments c m be computed

fÎom a single sample.

2.6 Decomposition of the Observable

In the reai world, processes cannot be rnodeiled as purely deterministic or

stochastic. Instead, one is faced with a mixture of both. CIearly, there are many factors

which prevent us to mode1 in a detexministic way. Most are due to either masurement

errors or systems that are simply too complex to be modeiled entirely deterministicaiiy.

According to Priestley 11981, p. 141 6'almost aU quantitative phenomena occirrring in

science shodd be treated as random processes as opposed to detenninistic functions."

The expected value of a random process may be computed h m some deterministic

model describing the expected behaviour of the series. However, this model wil l pmbably

not describe the series exactly as mentioned above. A stochastic mode1 may then be used to

account for the resulting lack of f i t It is thmefore convenient to decompose the observable

Nt) into a detemiinistic or trend cornponent &t) and a random or stochastic component e(t);

i.e.,

~ t ) = &t) + e(t) , V t e (a,-).

The mndom component e(t) may also be decomposed into two components:

whae s(t) is a statistically dependent (correlated) component and q't) is a statistically

independent (uncorrelateci) component The observable may then be represented in the

form

The statisticaily dependent component is often due to effects neglected or incompletely

accounted for in the deterministic mode1 defineci by @ Both random cornponents art

assurned to have a zero mean. This is enforced when the trend component is estjmated by

lest squares. However, due to the statistical dependence, there is a correlation among the

s(t) components. This statistically dependent component can be thought of as the residual

determïnistic part remainùig after removing the postulateci detedstic d e l . Thus, this

component is ofren referred to as a systematic emn or systematic effect.

Chapter 3 The Fourier Transform and Spectrum

3.1 Fourier Series and Integrals

It is weU known in matkmatics that any continuous, periodic bction can be

represented by an infulte series of trigonometric functions, called a Fourier series. If Nt)

is a function of period T, it can then be expressed in the form

where ai and bi are the Fourier coefficients comesponding to frequencyf;:. The frequencyfi

can also be expressed in terms of the namal or fundamentai fbquency f, asfi=&, where

fo=lB: Note that if angular frequencies (cu) are to be used, q (in radians per unit of t )

should be substituted for 2Mi.

Using the fact that the cosine and sine functions form an orthogonal basis over the

intervai (-272, T/2), the Fourier coefficients for di i = 0, ...,- are given by [Priestley,

1981, p. 1941

If Nt) is an even funçtion, i.e., Nt) = #-t), the bi coefficients are zero. When Nt) is an

odd function. Le., Nt) = -#-t), the ai coefficients are zao. Note that writing the constant

t e m in eqn. (3.1) as rather than as og d e s the expressions (eqns. (3.2) and (3.3)) for

the coefficients valid even for i=O.

For non-periodic functions, there is no such Fourier series representation.

However, accmding to Priestley [ 1981, pp. 198-2001, a new perïodic function may be

defined which is the same as the non-periodic one over a finite interval, Say, (-T/2, TL?)

but repeats itself and is thus per idc outside this interval. This new fûnction will have a

p e n d T and can now be represented as a Fourier series. By letting T-, the dimete set

of frequencies in the Fourier series becornes a continuous set of fkequencies; Le., an

integral. The non-periodic funceion can then be represented by the scwalled Fourier

integral which has the fonn [Riestley, 1981, pp. 198-1991

where the Fourier coefficients over the continuous range of frequencies are defïned by

2 oy) = T ~ @ ( t ) c o s Z R f t d t , Q f ,

This representation of a non-periodic funchon in terms of a continuous set of frequencies

holds only when the function is absolutely integrable over the infinite interval (-O, -)

[Riestley, 198 1, p. 2001; i.e.,

This happens when Nt) decays to zero as t goes to infimty.

So far only periodic and non-periodic determimstic functions have been considered

However, in practice one usuaily deds with random or stochastic functions (processes)

where the appIication of the above representations is not so apparent. Ciearly, stochastic

functions may not necessarïiy be periodic and thus they cannot be represented by Fourier

series. Furthmore, stochastic functions are not absolutely integrable since, by the

definition of stationarity, they do not decay to zero at infinity. It would then appear that we

cannot represent them as Fourier integrals either. Nevertheles, accordhg to Priestley

1198 1, p. 2071 it is possible to circumvent this problern by sirnply mcating the stochastic

process at, Say, -Tl2 and TE as done for non-periodic functions. ûutside this interval the

function is defuied to be zero, thereby satisfying the abwlutely integrable condition. As

long as the stochastic function is con~uous, it can be represented by the Fourier integral as

in eqn. (3.4) but with coefficients defïned by finite Fourier integrals using integration limits

(Ti2,-TI2) instead of (--,-); i.e.[Priestley, 198 1, p. 2071,

Unfominately, we cannot take the lirnit T+ as before since, by the pmperty of

stationarity, the above integrals would not be finite.

Although aU of the expressions for the Fourier series and integrais were given in

texms of trigonometric huictions, it is more cornmon to use complex notation for a more

compact representation of the series and integials. Assigning the cosine t a n to the real

component and the sine temi to the imaginary component, each trigonometric term can be

replaced by a complex exponential funetion using Euler's i o d a wronshteki and

Semendyayev, 1985, p. 4741

where j=d-1 is the irnaginary unit.

Using this notation, the Fourier series in eqn. (3.1) can be re-written as mestley ,

1981, p. 1991

where

Substituthg for ak and bk, using eqns. (3.2). (3.3) and (3.10).

T/2 1 a

Putting rhis in the Fourier series in the continuous form of eqn. (3.1 1) and letting T+

gives the s0i:aiied Fourier integrai over a continuous range of observations; Le.,

( [@(r) e - ~ z d dr for non-penodic functions

@(t) e--2?ft dt for stochastic functions -d 3.2 Fourier Transform

Given the Fourier integral representation of a non-periodic or stochastic funetion,

the transformation fbm Nt) to F(fJ in eqn. (3.15) is called the (direct) Fourier aansfom,

or the finite Fourier transfomi if deahg with stochastic functions. The transformation

fkom FV) to Nt) in eqn. (3.14) is cailed the inverse Fotuier transfom. Nt) and F m are

referred to as a Fourier &orm pair, denoted by Nt) o Fm. Note that the complex

conjugate fom is used in the direct aansfom and not in the inverse transform. In some

texts (e.g., Press et al. [198q), the conjugate form is used in the inverse ûmsform and not

in the direct transfom.

In practice one rarely deals with CO~MUOUS stochastic processes of infinite length

but d e r with actual discrete processes or discretely sampled data h m continuous

processes of finite length. Although such discrete samples are o h evenly spaceà in time

(or any other arpen t ) , this may not always be die case. Nevertheles, the application of

traditional Fourier nansfarm techniques requires the procases to be discretely and evenly

samp1ed This is because the trigonomeaic functions are not orthogonal over an unevedy

spaced domain.

For a discretely and evenly sarnpled stochastic process or data senes {Ntd,

i=0,1, ..., e l } , the discrete Fourier transform is obtained by approxima~g the Folmer

integral in eqn. (3.15) with a summation; Le.,

where n is the number of 'observations" (samples), & is the sarnphg interval and fk is one

of the frequencies belonging to the set of fkequencies estimable h m the discrete pmcess

(see below). Note a h that the summation index extends fîom O to n-1 (instead of 1 to n)

following the usual convention. If T = d t is the length of the data series, the discrete set of

frequencies are given by

where fo=l/T=l/(ndt) is the fundamental frequency. To make maners simpler, n is

assumed to be even (the data mies is truncated to an even number of points). This set of

integer multiples of the fundamental fiequency will be simply caiied cTourier" fraluencies

here because they are always used in the Fourier transfm and Fourier specmm.

By convention, it is only the b a l summation in eqn. (3.16) (widiout the At in

fiont) that is commoniy refmed to as the discrete Fourier transfomi, denoted by Fk for

frequency fk. The discrete Fourier transfomi OFï) is then defined by

The inverse discrete Fourier transfomi is obtained simiiarly by appruxima~g the integrai in

eqn. (3.14) with a summarion and substini~g for the d isæte Fourier naosform. This

gives

The discrete sampling of a stochastic process has an important consequence hown

as the aliasing effect, whereby some high frequency information will be lost or, more

precisely, hidden (aliased) in the lower fkquencies. This can be seen by examining the

exponen tial term in eqns. (3.17) and (3.1 8) as a func tion of the discretely sampleâ process

#(rd, i = -,...,O, where ti = i At and At is the sampling interval. Re-wrïting the

exponential function as

the effect of discrete sampling on each sine and cosine texm can be seen. For example,

substituting iAt for ti in the cosine term gives

The same cm be written for a new kquency f+M,

COS 21@f+Af)ti = cos(2IRfdf + Zia'&&) .

These two msine tums are quivalent only if2îrigfdt is an integer rndtiple of z This

, d e d the Nyquist or critical fkquency. occurs when At is an inîeger multiple offnr = 2dt

have fkquencyf. The same holds for the sine tanis. All frequencies outside of the

Nyquist frequency range ( - f ~ f ~ ) wiU "11 beased to (i-e., moved to and superimposed on)

kpencies in this range. If possible, At should be chosen srnaLi enough to avoid aliasing.

However, this requires a knowledge of the upper frequency limit of the information

contained in the process being sampled or, at lest, knowledge that only negligible

information exists beyond the Nyquist frequency and our willingness to neglect this

information.

There are some speciai properties of Fourier transf' that are of partïcular

importance. niese are summarized as follows (* indicates the complex conjugate

operator):

Nt) is red

Nt) is Maginary

Nt) is even

Nt) is odd

F(-- = FM* ,

F(-j) = -Fm* ,

F(-- = Fu) (i-e., F m is even) ,

F(-fj = -Fm (Le., Flf) is odd) .

Note that when de al functions, the senes of trigonometric t e m of cosines and

suies d u c e to a series of only cosine terms; Le., by eqn. (3.19) the sine terms are all zero.

In this case the Fourier sansform reduces to the so-called cosine transform

The foIIowing are some other properties of the Fourier transfm (fkom Press et al.

11992, p. 4911). Recall that #j(t)oFm indicates that Nt) and F(B are a Fourier transfomi

pair.

3.3 Fourier Spectrum

The representation of functions in temis of Fourier senes has a special physical

interpretation in terms of power (cf. Priestley [1981, p. 194-1951 and Ress et ai. [1992, p.

4921). Consider an absolutely integrable non-penodic function f(t). The total "power" of

Nt) is customarily defined by

00

Total power = #(t )2 dt . a

Substituang the inverse Fourier transfom in eqn. (3.14) for one of the M t ) gives

Interchanging the order of the integrals and substituting for the direct Fourier transforms

where F*V) denotes the complex conjugate of Fm. The total power can therefore be

e x p d eitha in temis of the integral of the original function or its Fourier transfomi;

1-e.,

w QO

Total power = 1 ( ~ ( r ) 2 dt = 1 i ~ ( f 112 df . a -0

This is known as Parseval's relation [Jenkins and Watts, 1969, p. 25; Priestley, 1981, p.

2011 or Parseval's theorem b s s et al., 1992, p. 4921. Note that the total power is qua1

to n tirnes the variance S.

It can be seen h m q. (3.34) that the total power is divided among a continuous

set of muencies in the representative Fourier inte@ Each term F(BI*@ represents the

contribution to the total power in Nt) produced by the components with frequencies in the

intend (f. f+a. The so-cailed power specnal density sO) for frequency f is thus defined

by

The plot of s(f) versus fiequency f is also called the power specmun, or simply the

spectrum. Theoretical power spectral density functions for some specid functions are

illusnated in Figure 3.1.

Constant - ACF Constant - SDF I l

Sine Wave - ACF

Exponenhd - Aff Exponential - SDF

-1 -0.5 O 0.5 1 O 0.5

1 Sine Waye - SDF

Figure 3.1: Autocorrelation functions (ACF) and power spectral density functions (SDF)

for some special functions.

0.5

(I*

- -

For periodic fimctions, the total power over the entire interval (-o., -) is infinite

mestley, 1981, pp. 195,205j. Aithough it is only needed to describe the power over the

fînite interval (-T/2, T12) in d e r to characteriz it for the entire infinite intemal, it is

usuaüy mon convenient to use the totai powa per unit of the over the finite interval. This

is obrained by dividing the total power by the period T; Le.,

Total power per unit of tirne -2 , 2 - r T I - ~ o t a i power (4,

The total power over (-T/2,T/2) is divided among the infinite set of discrete frequencies in

the representative Fourier series. The contribution s(fk) to the total power per unit of time k

of each T0wiei7 fkquency f k = ~ is calIed the spectral value for frequency fk and is

Similarl y, for stationary stochastic functions (random processes), the total power is

aiso infinite by the definition of stationarity (i.e., a steady state process h m t = -.. to =

requires infinite energy or power). Using again the truncation approach, stochastic

processes can a h be represented by finite Fourier integrals in the nnite interval(-272, TIZ).

The total power in this finite interval WU then be finite.

For bot. non-periodic and stochastic fuactions over a finite intaval (-Ta, T/2), it

is g e n d y more convenient to aiso use the pown per unit of time. As for penodic

functiom, the power pes unit time is obtained by dividing the total powa over the f i t e

interval by the length of the interval; Le.,

Total power (-5- , ) T T ~otaipowerpaunitoftime[-2 , 2 ) = T

Here s(B represents the power spectral density function. For a process of finite length T it

is defined by

The s p e c m denned above is a function of both positive and negative frequencies

and is cailed a "two-sided" s p e c d density function. However, one does not usually

distinguish between positive and negative frequencies. Moreover, when Nt) is Ral, the

Fourier transform is an even function; i.e., Ffl=F(-B. It is therefore customary to express

the specmim as a fiinction of only positive fkquencies. Such a spectnim is called a "one-

sided" spectral density function. Because the total power in the pmess must remain the

same, the spectrai values for the one-sided spectxum are defined as

For real Nt), FV) = F(-jj and

Hereafter, the one-side spectral density fhction wil l be used since only real Nt) will be

considerd

It is also convenient to nomialize the spectral values so that they express the

percentage of the total power or variation in the process contributed by each fhquency.

The normaiized spectral values 5*m are given by

A couple of important properties foi power specaa are obtained h m the properties

of Fourier transfomis. One of the most important is the invariance of the specaum to time

shifang. Given a pmcess Nt) shified by t,, the new Fourier aansform is

The spectrum s'(B for this process is then given by

which is identical to the specmim of the original series- Note that the constant exponential

term in eqn. (3.43) cancek with its complex conjugate.

The specrmm is not invariant with respect to time scaling, however. Intuitively,

expanding tirne effdvely resuits in s w g frequencies, and vice versa The relation

between two spectra with different time seales cm be obtained from eqn. (3.28). a v e n a

function Nat) which is scaled in time by a faftor a, the new Fourier aansform F'f) is, by

eqn. (3.28)-

This resuits in both a d n g of the frequencies as weU as the Fourier transfomi and

spectnim.

For discretely sampled, infinite length processes, the Fourier transform and spectrai k values are defined ody for the discrete set of ''Fourier" fkequencies fk = a k =

-n/2,...,n/2 (see discussion of discrete Fourier transform). The discrete fonn of Parseval's

relation for the total power in a process is obtained in the same way as for the Fourier

integrai except that the discrete Fourier transform is used instead. Foilowing the same

substitution and reordering of summations in eqn. (3.33) gives [Press et al., 1986, p. 3901

The individual spectral values s(fC) for the power spectral density hction are then given

by

The normaiized spectrai values are obrained by dividing by the total power as in eqn.

(3.42). For the diçcrete case. this gives

ln-l RealiEng that the variance d is the total power divided by n ($ = n ç ~ t i ) 2 ) , the

a n o d z e d specaal values can also be written as

Sample estimates of the spectrum c m be obtained by e v a l u a ~ g the discrete Fourier 1

transfomi for frequencies fk = O, ...,= and cornputhg the spectral values sud using

eqns. (3.48) or (3.49). It is important to note for later that this is equivalent to (i)

e v d u a ~ g the Folnirr coefficients ak and bk for the dimete hquencies fk using l e s t

squares estimation and (ü) computuig the (amplitude) specmun h m (ak* + bk2). For real-

valued functions, only positive frequencies need be considered because the negative

fiequency part of the specmim is the minor image of the positive part. Howeva, the

negative m e n c i e s will be aiiased as positive ones an& combined with the (identical)

positive ones. will result in spectral values twice those computed using eqn. (3.48), except

for the zero frequency. This gives the one-side specmim ratha than a two-sided spectrum.

The spectnnn computed in this marner is g e n d y referred a as the periodogram

[Priestley. 1981, p. 394; Ress et ai, 1986, p. 4211 and foms the basis of the least squares

spec-

3.4 Convolution and Correlation

Another application of Fourier transforms is in the concept of convolution and

correlation. Given two functions Nt) and Xt) and the5 Fourier aansforms F m and G(B,

we can combine these two functions together in what is calleci a convolution. For the

continuous case the convolution of Nt) and Nt), denoted @(t)*xt), is defineci by

[Bronshtein and Semendyayev, 1985, p. 5821

where ris thought of as an argument (time) difference or lag. The wnvolution theorem

then States that the Fourier transform of the convolution of two hinctions is equal to the

product of the Fourier transforms of the individual functions [ibid, p. 5821; Le.,

where the syrnbol o again signifies that the functions on either side are Foinier transform

pairs. The Fourier transfomi is used to go h m left to right while the inverse nansform is

used h m right to left

For discretely and evenly sarnpled processes Nt) and gti) , i = -n/2,.. .,n/2, the

discrete convolution is defied by

where the lags t i t i 4 are eveniy spaccd. The discrete version of the convolution theorem is

then

for fiequencies fk, k = O,...,n- 1.

Closely related to the convolution theorem in eqn. (3.51) is the correlation theorem

It cm be s h o w that the product of a Fourier transform with the complex wnjugate of

another Fourier tmnsfom can be reduced to the form [Priestley, 198 1, p. 2 1 11

where K is caîled the kemel:

In the conten of spectral analysis, the kernel K(r) represents the cross-covariance fiinction.

Multiplying the Foiiner transfonn by in own cornplex conjugate gives the autocovariance

funCaon C(r) (cf. Section 2.5) as the kernel; Le.,

Realizing that this multiplication gives the spectral value for fresuencyf, the covariance and

the spcctmm hinction can be expressed as a Fouriex transform pair; i.e.,

This is known as the Wiener-Khinchin theorem. Furthermore, the nmalized s p e c m

r(B is the aansfm pair of the autocorrelation funcbon R(t) (cf. Section 2.5) so that

When computing the convolution of two iùnctions care must be exercised to avoid

socaiied "end effects" or "wrap around effects" caused by assuming the fimctions to be

periodic. For example, when convolving a function wi th itself (Le., autocorrelation), data

h m the end of the series are effectively wrapped around to the beginning of the series

thereby fomiing a penodic function with period T. This can have adverse e k t s but can

be prevented by simply "padding" the data series with enough zeros to avoid any overlap of

original data. To estimate ail possible frequenties up to the Nyquist hquency (defïned in

Section 3.2), a data series of n points must be padded with n zeros to completely avoid any

wrap around affect. There is a nade off when dohg this, however; the more zeros

appended to the series, the pa te r the aors in the sample eshates of the Fourier

aansforms. See Press et al. [1992, pp. 5331 for more information on end effects.

These indirect expressions in temis of the spectnun are often used as the basis for

the efficient computation of autocovariance and autocomiation fundons using the FFI: It

will also be used as the basis for developing autocovariance fiinctions for unevenly spaced

data to provide objective a priori estimates of covariances and weights that account for

residual systematic effects in lem squares modelling. However, it must be adkd that

this indirea procedure gives the biased estimate of the autocovariance and autocorrelation

functions [Bendat and PiersoI, 197 1, pp. 3 12-3 14; Priestley, 198 1, pp. 323-3241.

3.5 Fast Fourier Transform

Any discussion of the Fourier transform would not be complete without mentionhg

the secalled Fast Fourier Transform (FFT). Although the term is often used

synonymously with the Fourier transfomi itself, it is reaiiy only a numerical algorithm used

to compute the discrete Fourier transfixm (Dm in an exmely efficient manner. The

algorithm, popdarized by Cooley and Tukey [1965], revolutionized the way in which the

DFT had been used. Up to that tirne the DFT was restricted to only srnall data sets. W i h

the advent of the FFT aigorithm. however, it was quickly employed in a multitude of

applications.

nie basic idea behind the FFT is a bisection and recombination process. F i t the

data is repeatedly bisected into pairs of points by recursively dividing the data into odd and

even numbered points. The Fourier transfoms are then computed for each of these pairs

of points and subsequently recombined to foxm the Fourier transform of the entire data

series. Because the Fourier transform of a pair of data points is a trivial and very fast

computation (no multiplications are needed), the algorithm results in a dramatic increase in

computational efficiency, especially for large data sets. The number of (cornplex)

multiplications involvd in the direct evaluation of the disate Foinier trandorm is of the

order of 9 whereas the number of such operations in the FFT aigorithm (in the

recombination of the individual transforms) is only of the Ofder of n log2n [Press et ai.,

19921. This general strategy was fîrst used by Gauss to d u c e the computationai effort in

determinhg planetary orbits and also derived by as many as a dozen others since (see

Brigham [1974] and Bracewell[1989] for more information).

The main limitation of b t h the discrete Fourier transform and its FlT algorithm is

that the data must be equaiiy spaced. The expression for the Fourier coefficients, and thus

the Fourier transfomi, are vaiid only for equdy spaced data. Moreover, the FFT algorithm

uses certain properties of the sine and cosine functions for eveniy spaced data to reduce the

number of temis that ne& to be evaluated For the investigation of systematic effects

which can be functions of many different kinds of arguments that are usually very

irreguiarly spaced, this precludes the use of the FFT, at least in the computation of the

discrete Fourier transfm A similar problem also arises when there are large gaps in an

othenvise equaily spaced data series.

To circumvent the problem of unevenly spaced or "gappy" data, interpolation

schemes are someeimes used where the original data are interpolami to give an evenly

spaced series. This then ailows one to use naditional techniques such as the FFï.

However, the accuracy of the interpolating function to represent the original data series

depends on the form of the interpoia~g function, the smoothness of the original data Senes

and the presence of large gaps in the data. This presents a dilemma since in order to

properly interpolate the &ta we must have a good howledge of their behaviour, but the

lack of this knowledge is usually the reason for computing FFTs in the £kt place. Another

problem with interpolation is that in the presence of large gaps, they often result in

disastrous results.

A second limitation of the FFï' is that the number of data points to be transformed

must be a power of 2 for the FFT to be most efficient Altemate and mixeci mdu<

formulations of the FFî aiso exin but they are much less efficient. The conventional

method of deaüng with a number of points that are not a power of two is to again ''pi" the

data series with enough oeros to obtain the required number of points for the FFT. This

clearly infiates the number of points to process thereby increasing not only processing time

but also storage requirernena. It is most inefficient when dealing with large data sets. In

these cases. one usudy only takes the fnst power of two number of points and omits the

rest More importantly, zero paddllig also increases the ermr in the FFI' with respect to the

continuous Fourier transform,

One more limitation of the FFT is that it is restricted to only the set of 'Fouriei'

fre-quencies. If fkquencies other than these standard ones are present, a phenornenon

lmown as spectrai leakage can degrade the resuits . To compensate for this, so-called

window functions are employai to reduce this leakage by convolving a tapered Gaussian-

like function with the data senes in the Fourier transform. For more on window functions

see, e.g., Riestiey [l98 1. Chapter 71 and Press et al. [1992, Chapter 13-41,

3.6 Other Transforms

The preceding developrnents have been based on the use of Fourier (trigonomeaic)

series to approximate functions and stochastic processes. The advantage of using Fourier

series is that the periodic ternis are usually easier to interpret physicaliy. Nevertheless,

other approximation or basis functions can be used.

One popuiar alternative approximation hinction is the socaiied "cas" hinction

which forms the basis of the Hartley transfm m e y , 1942; BraceweU, 19861. This

function is defined as

and is used in place of d2@t = cos2@ - j sin2m in the usual Foimer expressions. Note

that the difference between the two is that the Fourier expressions separate the cosine and

sine ternis while the Hartiey expressions combine them.

In spie of the different functions used in the Fourier and Hartiey transforms, they

are simila, in shape. In fact, the Fourier transform can be deduced b m the Haniey

tramform. aithough this is wnsidered unnecessary because either transfona provides a pair

of numbers at each fiquency that represents the oscillation of the series in amplitude and

phase @3raceweU, 19891. Moreover, the amplitude and phase spcctra obtained from either

nansfonn are identical, although they are derived in a slighdy different manna [ibid.,

19891.

As for the Fourier aansforrn, Braceweil[1986] has also developed a fast Hartley

transfomi in much the same way as the FFT. The advantage is that the fast Hadey

transform has k e n shown to be nvice as fast as the FFï and uses haIf as rnuch cornputer

memory [O'Neill, 19891.

Chapter 4 The Least Squares Transform

4.1 Introduction

A significant limitation of the traditional techniques for the estimation of

autocorrelation fwictions, either directly or indirectly via the inverse of the Fourier

spectrum, is that they always require the data to be equdy spaced in the argument.

Although the data might be evenly spaced with respect to some basic sampling parameter

such as time, it will generally not be evenly spaced with respect to other parametas that

may better characterize the behaviour of any systematic effeca tu be modelled by

correlation functions. Som typical parameters that might be used to mode1 such systematic

effects in geodetic problems include spatial distance, satciiite elevation angle, atmospheric

temperature, temperature gradient, pressure, etc.; cf. Vanilk and Craymer [1983a,b],

Craymer [1984; 19851, VmZek et al. [1985], and Craymer and Vaniiek [1986] Clearly it

would be very diffcult to obtain a data senes evenly spaced in even some of these

randody fluctuating parameters.

Other reasons for seeking altemative techniques are conmed with the limitations

of the discrete Fourier transfomi and FFI' describeci in the preceding chapter. These

include the use of ody the set of standard 'Fourier" muencies (infeger multiples of the

fundamental fkequency), and the requirement of 2n data points for the FFI' algorithm. Ui

addition, a detenninistic mode1 is often es- and rernoved h m the data prior to any

spectral analysis. Traditional spectral techniques do not consider any interaction or linear

dependence (correlation) between the a priori determinisec mode1 and the implied periodic

components modelled in the spectrum and in the comlation hinction. M m v e r , the data

cannot be weighted in the Foinier d o m computation in accordance with their assumed

probabiiity density fiuiction. Thus, some observations with relatively large random errurs

will be treated the same as other observations that may be many times more precise.

The aim here is to fornidate a more general aansform that is capable of handling

such unevenly spaced arguments. The transform is based on the least squares specaum

computation developed by VaniRIek [1%9a; 197 11 and is r e f d to here as the least

squares rransform and its inverse. Note that this l e m squares approach is developed here

for real-valueci data and, consequently, positive frequencies. It cannot cope with complex

data or negative frequencies, which are useful in distinguishing between prograde and

retrograde motions.

4.2 Matrix Form of Fourier Transform

The basic fom of the least squares transfomi can be derived by fifit expressing the

dismete Fourier transform (Dm in terms of matrices of complex exponentiai fumions.

Rewriting eqn. (3.18) in manix fomi gives (the superscript "c" denotes a complex matrix)

w here

Note that the transpose in eqn. (4.1) is the complex conjugate transpose for cornplex

matrices (see Golub and Van Loan [1983, p. 9 1); i.e.,

This matrix fonn of the discrete Fornier transform can be written for each of the discrete

''Fourier" frequencies in eqn. (3.17).

Combining ail fhquencies together gives the simdtaneous transform for a i l the

standard Fourier frequencies; i.e.,

w here

The transpose in eqn. (4.5) again indicates the complex conjugate transpose where

Note that AVk in eqn. (4.1) is the k-th columa of AC coftesponding to the specinc

frequenc~ fk-

The inverse discrete Fourier transform expresses each observation Nti) in tnms of

the Fourier traasforms Fk for ali of the discrete T0uriei7 frequencies fc = k/(ndt),

H l , ..., el. This a n also be written in ma& form as for the direct transfomi. Rewriting

eqn. (3.19) in rnatrïx notation gives

where

Combining a l i observations together gives the sirnultanmus inverse uansfonn; i.e.,

where AC is defined as in eqn. (4.7) and @ is denned by eqn. (4.3). Note that the design

matrix is not transposed in the inverse transfom and a factor of Iln is included as in the

cornplex form. Expanding this in ternis of the Fourier msfonns for the individual

Totrrier" fiquencies gives

Befare developing a more generaï least squares fom of the a b e transfm, it is

necessary to replace these complex expressions with their reaI-value- nigonomenic forms.

It will be show later that this i s because, for unequally spaced data, the real and imaginary

components can, in general, no longer be treated independently of each other. Using

Edex's formula (eqn. (3. IO)), the discrete Fourier transfonn in eqn. (3.18) becomes

and the inverse discrete Fourier transform is

Note that the sine term is zero for the zem frequency component ( H l ) in the above

expressions. Realizing that the red (cosine) and irnaginary (sine) temis are two separate

quantities that are independent of each other, the complex expression can be rewritten as

two separate real expressions for each tem. That is, for the reai term,

and for the irnaginary tam,

The discrete Foinier msform in eqn, (4.1) can now be expressed in real matrix

notation using eqn. (4.1). with separate columns in the design matrix A for the r d

(cosine) and imaginary (sine) temis. The transform is then aven by eqn. (4.1). where FCk

and AYk are replaced with Fk and Afp respectively, which are defined as

Note that for zero frequency (ka), Im(F,)=O and ai l the sine temis in Af, are also zero, so

that

The sirnultaneous direct and inverse Fourier tnnsforms for a i l the "Fouriei' frequencies are

then given by eqns. (4.5) and (4.1 l), respectively, with F and AC replaced by,

respectively,

Note that there are n observations and ody n-1 coefficients to solve for.

4.3 Least Squares Transform

A more general least squares transfomi (LST) cm be obtained h m the above

ma& form of the discrete Fourier msform (DFT) by reaiizing that the DFT and its

inverse are equivalent to least squares interpolation or approximation using aigonometric

functions (Le., Fourier series) as the basis functions (see, e-g., Vmiek and Krakiwsky

[ 1986, Chapter 121 for a detailed exposition of least squares theory). Specifîcaiiy. a vector

of observations 4 can be approximated in tems of a Fourier series by eqn. (3.1). which

can be written in matrix notation as

where

is the vector of Foinier coefficients to be estimaestimated and A represents the basis

(trigonometric) functions as de- in eqn. (422). Note that for fo=O, there is no

imaginary tem and thus no bo coefficient The Fourier coefficients x can be estimated by

solvhg for them using the least squares mInimization criterion (cf. Vaniiek and Krakiwsky

[1986, pp. 204-2071). The solution is given by

where N =A* A is the nomai equation coefficient rnatrix.

Note that in the above equation AT# is the manix fonn of the (Smultaneous)

discrete Fourier tmsfonn in eqn. (4.5). T'us, the l e s t squares transfomi for aIi

frequencies simultaneously is given by eqn. (4.5) and the transform for each frequency fk

by

where Ak that part of A corresponding to only frequency fk.

The estimated Fourier coefficients in eqn. (4.25) can then be written as

Substituting this in eqn. (4.23) gives the estima& observations

a = A N - I F ,

which represents the simuitaneous inverse least squares aan~fom for a l l Frequenties. The

individual observations Rti) are then given by

where Ap represents the i-th row of A corresponding to time ti.

The conventionai Fourier aansforms are just a s p e d case of these more gewral

Ieast squares definitions for equally weighted and equally spaced data. Although the direct

least squares and Fourier transfomis are equivalent by definition, the quivalence of the

inverse aansfoms is not easy to see h m the ma& expressions. This quivalence can be

shown by examining the elements of Pl. Realizing that the Fourier expressions are valid

only for equally spaced data and the discrete set of "Fourier" fiequencies, it can be shown

that the columns of A form an orthogonal basis under these assumptions. The elements of

N (summations of trigonomemc products) reduce to

n-1 O for k=I=0 or n /2 z ( s i n 2 ~ ~ t ~ sin2 lcfitii) n/t for ~ = I * o or n/2 , id O for k d

Substituting these in N-1 in eqn. (4.28) and expanding in tems of the Fourier transfom

for the individual frequencies gives

The difference between this and the inverse Fourier transform in eqn. (4.1 1) is the use of

n/Z in place of n for non-zero fkquencies (n is assumeci to be even, 0th- n/2 is

nuncated down to the nearest integer). This is because for real data the transforrn for

negative fiquencies is identical to that for positive frequencies. The wIumns of A

comsponding to these fiequencies wii i be identical thus making N singular when

simuitaneously estimating a i l fkquencis. Including only the positive frequencies wiLi

irnpiicitly account for the identical response for both negative and positive frequencies,

thereby effectively doubling the lest squares transform with respect to the Fourier

transform (i.e., it gives a transfonn which resdts in a one-sided specnum as derived in the

next chapter) . Note that the Nyquist fi-equency (at k = a ) is also excluded h m the

summation since this is aliased with the zero frequency.

It is important to realize that for unequally spaced data the inverse lem squares

transfomi in eqn. (4.28) cannot in general be expresseci as a summation of independent

contributions h m individual fkquencies. This is because N in general contains off-

diagonal elernents between frequencies and even beween the sine and cosine components

for the same fkquency; Le., these Fourier components are mathematicaîiy comiated with

each other (Le., they are no longer orthogonal or linearly independent).

4.4 Weighted Least Squares Transform

The above developmenn have irnplicitly assumed the observations to be equally

weighred A more general form of the least squares tmnsforms cm be derived by

w e i g h ~ g the observations using their associateci covariance ma& C+ This also dows

one to mode1 any known correlations among the obse~ations. The generai expressions for

a weighted least squares interpolation or approximation are given by (cf. Va&& and

Krakiwsky [1986, pp. 204-2073)

where N = AT P A is the normal equation coefficient maaix, u = AT P #, is the normal

equation constant vector and P = C+-1 is the weight matrix of the observations.

Foiiowing the same development as for the unweighted (i.e., equaiiy weighted)

Ieast squares transfonns, the more general weighted Ieast squares transf~~lll for all

frequencies sirnultaneously is given by (cf. eqn. (4.26))

and the transfom for each individual frequency fk by (cf. eqn. (4.1))

Using this in the least squares estimation of the Fourier coefficient in eqn. (4.34) and then

substitu~g into eqn. (4.35) gives the inverse kast squares aansfom (cf. eqn. (4.28))

The individual observations Mti) are then

Although the symbolic fomi of these expressions are identical to those for the unweighted

inverse transfm in eqns. (4.26) and (4.271, N and F are de- differently (they include

the weight rnaaix P). Note that the inverse transfm is essentially just a least squares

approximation of @ in terms of a Fourier series.

As stated at the end of Section 4.3. it is not possible in g e n d to separately

esàmate the individuai Fourier transfom values for different £requencies because of the

possible existence of mathematical correlations (non-onhogonality) among the Fourier

components (hg functions) due to unequal data spacing or correIations among the

observations. If, however, the observations are equaily spaced, equally weighted and

uncorrelated (Le., P = 4, and the set of "Fourier" frequencies are used, the normal

quarion ma& becomes a diagonal (Le., N = diag(n, n/2. fi,. . .) and the direct and

inverse least squares transfomis become identicai to eqns. (4.26) and (4.33). respectively,

and are thus equivalent to the standard Fourier ones. The Fourier transfonn is thus just a

special case of the least squares transfona

An attractive feature of the lest squares tmsform is that the covariance ma& for

the Fourier coefficients and the inverse least squares transfonn are pmvided by the lest

squares theory as by-products of inverthg the normal equation manix N (cf. Vani.ek and

Krakiwsky [1986, pp. 209-2101). The covariance matrix for the estimated F o e

coefficients 9 is given by

while that for the inverse transfm (mterpolated/approximated obsewations) is

cg = A Q A T .

It is recalled that oniy frequencies up to, but not including, the Nyquist fkquency

should be included in the Fourier series in order to avoid singularities in N due to the

aliasing effect In addition, if the data are e q d y spaced, only the set of standard

T0urief7 fiequencies should be used (see Section 3.2). Moreover, if the data are reai,

only the positive Fourier fhquencies shouid be included (see pmperty in eqn. (3.23)).

This then allows fm a total of n-1 terms (a-1 cosines and n/2 sines) to be estimated fiom

n observations, which gives a nearly unique solution for the Fourier coefficients and

enables the observations to be repmiuced exactly using the inverse transform.

In addition to a ~ ~ e p ~ g uneqdy spaced data, another advantage of the least

squares transforms are that they are not restricted to only the set of standard Fouria

fiequencies fc = Rl(n At) = k/T for ka, ..., *1. Any set of frequencies in the range (O, f ~ )

can be used in the expressions. However, only a m;ucimum of n/2 fiequencies (n Fourier

coefficients) can be estirnatecl sirnultaneously fkom only n observations. Moreover, some

serious repercussions can aiso arise if the selected fkquencies result in some of the Fourier

components (hg functions) becoming nearly linearly dependent with each other, thereby

producing an ilkonditioned or near singular N. To avoid such ill-conditioning it becornes

necessary to either select a different set of hquencies to be estimated (e.g., equdy spaced

frequencies) or simply neglect the correlations in N (i.e., the off-diagonal blocks) and

esthate the inverse least squares transform separately for the individual frequencies using

eqn. (4.39).

Another problem in dealing with uneqdy spaced data is that the Nyquist

fiequency is not weil defined, if at ail. It was thought that, because a single cycle of a

periodic function can be defined with only 3 points, the smaiiest time interval of a mplet of

adjacent points would represent the smaliest pend which can be estimateci. Care would

also be need to ensure that no pair of points in the triplet are so close together that the niplet

is essentialIy only a pair of points for ali practical purposes. In practice, however, this

triplet interval does not appear to defhe a Nyquist fiequency. As WU be shown in the

numericd tests of Chapter 7, spectra wmputed to frequencies well beyond this implied

Nyquist fresuency do not exhibit the expected mirror image about any Nyquist frequency.

4.5 Effect of Deterministic Mode1

So far it has been assurneci that the original data is stationary and can be modeiied

completely by a Fourier series. In general this is hardly ever the case. It is more common

to first remove the non-stationarity by modebg some hown a priori deterministic trends

using, e.g., least squares fitting and to analyse the residual (stochastic) series using the

above techniques. The problem, however, is that there may be linear dependence beh~een

the deterministic d e l and the periodic components in the Fourier series (the stochastic

mode1) which may significantIy affact the Fomier transfbrm and spechum.

To account for such effects, it is necessary to refomiulate the preceding

developments to accommcxiate both the detenninistic mode1 as weil as the stochastic mode1

(periodic Fourier senes components) in the esthnation of a least squares fransfom

Partitionhg A and x,

the data series (observation) vector 0 is modelled in ternis of both deterministic and

stoc has tic (Fourier series) & components as

For the de temiinistic d e l , AD is the design ma& and XD is the parameter vector to be

estimated, and for the stochastic (Fourier series) d e l , As is the ma& of cusine and sine

basis functions as defined in eqn. (4.22) and xs is the vector of Fourier coefficients to be

estimated as defined in eqn. (424). The aim is to account for the effect of estirnating XD in

the estimation of xs.

The weighted lest squares estimates of the combined parameter vector î. and the

appmximated observation vecmr ) are given by eqns. (4.34) and (4.33, where the

matrices are defined as above. Substitu~g the above partitioned forms of A and x into

these expressions gives

w here

Although, for stochastic modeIIing, we are ~ a U y only interested in k it is

necessary to account for any effect of the d e t e d s t i c mode1 on the estimation of & by

PD. This is obtained by making use of some well-known matrix identities in the evaluation

of Ifs. Specificaiiy. the inversion of the n o d equation maaix N can be written as

Wamiek and Krakiwsky. 1986, p. 281

w here

Substituting into eqn. (4.45) and gives for%

where the so-called "reduced" normal equation and constant vectur are

Defining the "reduced" weight matru< P., whicnch accounts for the effect of the

deterministic d e l , by

the normal equations in cqn. (4.58) can be wristen in the same general fomi as that without

the deterministic model; i.e.,

where

The simultaneous least squares transforrn F* (for aii fquencies simultaneously)

which accounts for the deterrninistic mode1 is then defined in the sarne manner as in eqn.

(4.36):

F* = A$ P* q .

The tmnsform for each individual fiequency fk is then (cf. eqn. (4.37))

Similady, the inverse transform for a l l observations is defined by eqn. (4.38), using the

reduced f o m of N and F, as

and for individuai observations &ti) by

The expressions for independently estirnated frequency components are simply obtained by

ignoring the offdiagond ternis between different frequencies in N* and P*

When there is no deterministic model, AD = O, P* = P and the above expressions

reduce to the same forrn as in the previous section. Note that the weighted inverse

transfomi is essentially just a weighted least squares approximation of # in temis of the a

priori deterministic mode1 and the individual periodic (Fourier series) components.

4.6 Vector Space Interpretation

The least squares aansform can be more elegantly interpreted using the concept of

Hilbert spaces and commutative diagrams using the language of functional analysis. The

fundamental component of functiond anaiysis is the space, in which we want to work.

The elements in a space c m be real numbers, complex numbers, vectas, matrices as well

as huictions of these. Here we consider the more restrictive case of vector spaces

consishg of sets of vectors which can be visualized as positions in the space. A brief

review of functional analysis as it applies to the geomeaical interpretation of the least

squares rransform is given. For more on functional analysis see, e.g., Kreyszig [1978].

There are various classifications of spaces. The most generai type of space is the

rnetric space in which the concept of a distance (or menic) p(*y) between two eiernents x

and y in the space is defined. A nonoed space is a memc space in which a nom II*II may be

induced as the distance h m the ndï element. The norm IMl of a single element x is just its

length p(x,O). A Hilbert space is a normed space in which a scalar (or inner) product

ap for a pair of elements x and y rnay be induced by the relations

There are many ways of defining a scalar product For vector spaces of f i t e

dimension the most common is the simple linear combination of vector elements; Le., for

vectors x and y ,

For compact vector spaces the analogous form of the scalar product is

A more general definition of the discrete scalar p d u c t , and the one used here. is the norm

defineci by

where P. the weight maaix for the v e c ~ space, is g e n d y the inverse of the covariance

ma& of the vector elements. This curresponds to a generalization of Euclidean space with

metric tensor I, into a Riananian space with metric tensor P. Note that for compact

manices, the vectm and manices wiil also be compact, and contain con~uous functions.

An interpretation of basic least squares theory in terms of functional analysis is

aven by VaniEek [1986]. The theory is interpreted using commutative diagrams which

desmïbe the various transformations between probabilistic (Hilbert) spaces. The same

diagram can be used to interpret the least squares transfom In this diagram @ is the

observation vector belonging to the observation space @, C4 is the observation covariance

matrix (not necessarily diagonal) defining the scalar product (and nom and distance) in this

space. x is the parameter vector of Fourier coefficients to be estimated belonging to the

parameter space X and A is the design ma& tmnsforming the observations to the

parameters, which contains the sines and cosines functions (basis functions).

The commutative diagram is set up by first defining the transfomation (i.e., the

observation equations) #=Ax from the parameter space P[ to the observation space @. The

weight matrices Px and P@ define the nansfonnations to the dual parameter space and

dual observation space @*. respectively. The transfomation h m the dual observation

space @* to the dual parameter space P[*. is defined by A ~ . Asslmiuig the design matrix

A and covariance ma& Ce are known, the rernaining transformations can be obtained

h m the commutative diagram using the following steps.

1. Pq, = C#-'

2. pX = A T P ~ A cX = P E I

3. A- = C,AT P# = (ATP* A)-' AT Pf

4. F = AT P#

These steps are Uustrated in Figure 4.1. Here, F is defineci sIightly differently ththan in the

pRceding developments. It repmsents the transfomi operator that acts on the observations,

and wt the entire transform itself as dehed in Section 43. Sindarly, F-1 is the inverse

operator.

It can be seen h m ihe commutative diagram that the Ieast squares Fourier

~ s f o m F is a trausfbrmation h m the observation space 8 to the dual parameter space

via the dual observation space 9'. The inverse least squares Fourier transform F-z is

then deriveci by proceeding fkom the dual parameter space to the observation space

via the parmeter space X.

Observation Space

Observation Space

Figure 4.1: Commutative diagram for the direct and inverse least squares ~ s f o r m ,

where F denotes the direct transform and F-l the inverse transfom.

The design matrix A contains the trigonometnc functions definiog the Fourier saies

representation of the observations. The individual sine and coshe terms (columns of A)

fom a basis for the obmaaon space. For the standard Fourier transfonn, the data are

equally spaced, equaily weighted and uncmlated so that the columns of A form an

orthogonal basis. The norrnai equation ma& N =AT P A then becornes a diagonal

ma& a s does the covariance rnatrix of the parameters. In the more generai least squares

aansfomi, the base functions are not necessarily orthogonal, although, in practice, this is

usually the case even with unequally spaced data.

4.7 Applications

The above least squares transfomi can be applied in the same mariner as the

traditional Foinier one, with the added advantage that they can be used not only for e q d y

spaced data senes, but also for unequaiiy spaced senes and for any arbitrary set of

fkequencies. One of the most important applications (to be discussed in the next chapter) is

the detdnat ion of the power specaal density fa unequaliy spaced data that also accounts

for a detemiinistic model. In this case there is no need to determine a frequency response

function for the detenninistic model in order to remove its effect f h m the specaum of the

model residuds. The correct spectnim is obtained directty when the detenninistic model is

acwunted for in the formulation of the spectrua

Another important application of the lest squares transfomi is the indirect

estimation of autocovariance/aut~~ofielation functions using the correlation theorem (see

Chapter 6). Instead of transforming the effect of al l the spectral values, a s m o o t k

autocovariance function cm be obtained by uskg only the significant specaal values.

Because these significant spectrai components are not likely to be evenly spaced, it is

necessary to use the inverse least squares transform to convert hem into an autocorrelation

function.

The inverse least squares tram$ann can also be used in data series approximation

and interpolation problem. In these applications the direct Fourier transform is used to

estimate FourÏer series coefficients, which are then used in the inverse t rans fm to

approxirnate or interpolate the original Senes. The degree of smoothing of the Onginai

series can be increased by induding only fiequencies corresponding to highiy significant

Fourier coefficients (or specaal peaks).

Chapter 5 The Least Squares Spectrum

5.1 Introduction

As dimissed in the previous chapters, traditionai mtthods of detemiining power

spectral density and autocorrelation functions are significantiy limited in their application

because they always require the &ta to be equally spaced in the argument. Other reasons

for seeking alternative techniques are concénied with the limitations of the discrete Fouier

transfomi and FFI' commoniy used to generate spectra as weli as autocorrelation functions

(ûansfomied from the spectrum). These include the use of only the set of "Fouriei'

fkquencies (integer multiples of the fundamental fresuency), and the requkment of 2n

data points (for the FFT algorithm). In addition, the aaditional techniques do not consider

any interaction (correlation) benveen the determinisac mode1 and the implied periodic

components modelled in the s p e c m . Moreover, the data cannot be weighted in the

transform computation in accordance with their assumai probability density hinction.

Thus, some observations with relatively large random errors will be weighted the same as

other observations that may be many times more precise.

Traditional methods of cornputhg power spectral density functions h m unequally

spaced data have often been based on interpolation or approximation. That is, the original

unequaily spaced data series was interpolated or approximated to an equally spaced series

for which the standard Fourier techniques could then be applied The problem, however,

is that this approach really mates a new data series that depends on the smoothness of the

original series, the presence of data gaps and the subjective choice of the interpola~g or

appmximating fiinction. The interpoIation also tends to smooth out any high frequency

components of the original data Senes.

To overcome these limitations and difficulties, VanRIek [1969a] developed a method

of spezûum computation based on least squares estimation. This method was M e r

developed in V a e k 1197 11, Steeves [ 198 11 and Wells et al. [1985] and forms the basis

of other Simüar techniques in slightly different f m promoted by various authors since

(e.g., Rochester et ai. [1974], Lomb [1976], Ferraz-Meilo [1981], Scargle [1982], Home

and Baüunas [1986]). In this Chapter, the same basic lest squares spectnim is

reformulated in ternis of the newly developed least squares transform. A new

"simdtaneous" spectrai estimation procedure, somewhat simila. to that used by Rochester

et al. [1974], is also developed.

5.2 Matrix Form of Fourier Spectrum

Before giving the expressions for the least squares spectrum, the Fourier spectrum

is first expresseci in matrix fom. This is done by simply using the matrix expressions for

the Fourier transfomi (eqns. (4.9) and (4.21)) in the definition of total power (eqn. (3.47))

and the individual Fourier spectral estimates (eqn. (3.48)). Parseval's relation in eqn.

(3.47) can then be written in mabtix notation as

1 Total power = eT# = n F=F .

where

The individual spead components for the twesided power spectral density function (eqn.

(3.48)) are then given by

The one-sided spectral density function is twice the two-side function and is defined by

1 1 - F ~ T F R = $~(f~)12 for k = O n

~0%) = (5.3 L F k T ~ t = ; I F ( ~ ~ ) I ~ f o r k = l ... n12-1 n

5.3 Least Squares Spectrum

The Least squares specmim was originally developed by V m i e k [1969a; 19721 (see

also Steeves [1981] and Wells et al. [198q). The expressions for this form of the least

squares specaum (referred to here as the '%onventional" form) can be developed in terms of

the (unweighted) least squares tiransform. First, the total power is given by

Total power = #T @ .

Substituting for the inverse least squares transform in eqn. (4.28) results in

Total power = @T # = FT N-1 F .

Note that, g e n d y , the total power cm not be expressed as a sum of individual

contributions h m the different fkquency mmponents. As with the inverse least squares

nansform, the problem is that with unequally spaced data, N is not a diagonal matrix

because the Fourier components (hg fwictions) are not orthogonal to (lineariy independent

of) each other. As explallied above, this problem is avoiâed by simply examining one

fhquency at a time, independently (out of context) of the othm. This is quivalent to

ignoring the iinear dependence between different fiequency components in N and amomts

to d e m g the spectnun as the independent contribution of each fkquency component to

the total power.

FoIlowing this approach, the s p e c d component sud (for the one-sideci l w t

squares power spectral density function) is defineci by

where Nk is the k-th diagonal block of N correspondhg to frequency fk. The normalized

spectral values T V , are then

The nomialized specmim represents the percentage of variation in the original data Smes

independently expiaineci by each s p e d component. In its basic phüosophy, this

corresponds to the ~2 statis tic in regression analysis m p e r and Smith, 198 1, p. 331.

One of the most signifiant advantages of the l e s t squares spectnun, other than

handhg unequaiiy spaced data, is the ability to estimate spectrai components for any real

(arbitrary) fnquency, not just the set of 'Fourier" fiequencies. The expressions in eqns.

(5.8) and (5.9) essentiaiiy provide continuous estimates for any set of fkpencies. The

usual procedm is to take a set of equally spaced frequencies between zero and the

e s tk t ed Nyquïst or maximum frequency (note that the Nyquist frequency is undefirred for

unevenly spaceà &ta as discussed Section 4.4). The pnxise fkquency location of

signifiant peaks cm then be detennùied by "mrning" in on that frequency area of the

specaum. This allows one to locate mencies for significant peaks to any resolution,

within the Iimits of the &ta sampling.

5.4 Weighted Least Squares Spectrum

The more general weighted least squares power specmun is obtained in a similar

way except that the general (weighted) least squares transfom are used in the above

developrnents. In this more general situation of an observation weight marrix, the total

power is defined by the weighted sum of squares as

Total power = #T P # , (5.10)

where P is the inverse of the observation covariance ma&. Substituthg for @ using the

weighted least squares inverse transfomi in eqn. (4.38) and n o ~ g that A= P A = N gives

Total power = #TP # = FTN-1 F. (5.1 1)

Vmiek [1969a] defines the spectrum as the independent fiequency contributions to

this total power (cf. eqns. (5.8) and (5.9)). That is, each frequency component is

estimated independently, or out of context, of the others. Steeves [1981] extends this

approach by incorporating the weight matrix (P) of the observations. The independent

estimate of each spectraI component is then obtained using the weighted lest squares

transfomi h m eqn. (4.36) in the specaal estimates given by eqns. (5.8) and (5.9). whae

the weighted normal equation matrix Nk for the k-th specaal component is defined by

This type of spectrai estimation is referzed to here as "independent" or "out-of-context"

spectral estimation.

An alternative approgch to least squares spectral estimation ain be deweloped in

which aii spectral components are estimated sirndtaneously; Le.. in the context of the

others king present This appmach takes into account the non-onhogooality (mathematical

correlations) between the specuai components. It is effectively equivalent to the

geometrical projection of the total multidimensional quadratic fomi representing the total

power, ont0 the subspace for each individual spectral component This is analogous to the

way in which quadratic fonns and confidence regions are dehed fm station coordinates in

geodetic networks. This estimation method is developed by first realizing that in eqn.

(5.1 1) for the total power the inverse of the normal quation matrix N-1 is equivalent to the

covariance matrix for the simuitaneously estimated Fourier coefficients (cf. eqn.

(4.40)). The total power can then be wrinen as

Total power = FT Cp F . (5.13)

Substituting for the weighted least squares transform in eqn. (4.36). the total power can be

expressed in te- of the estirnated Fourier coefficients:

Total power = $T Ccf-l/: .

The weighted edeast squares spectruru is defined as the contribution sud of the

individual fkquency componena (Fourier coefficients) a> the total power. That is, the

quadratic fomi of the estirnateci Fourier coefficients for individual frequencies is

where Cpk is the k-th diagonal block of covariance matrk C*. Substitu~g back in the

weighted least squares transform in eqn. (4.37) fot individual £kquencies, gives the

weighted least squares specaal values

which account for any non-orthogonality (mathematical correlations) among the different

specuai components. Note that Cgk is not the same as Nk-l in the expression for the

independently estimateci (out-of-contutt) l e s t squares specaum. Using the k-th diagonal

block from Cgk is the same as extracting the k-th diagonal block £kom N-1 instead of from

N as in the conventional expressions (cf. Steeves [1981]). Thus, eqn. (5.16) may also be

written as

The nonnalized spectral value S(&J for frequency fk is obtained by dividing by the total

power, i.e.,

This type of spectral estimation is xferred to hcre as "simdtaneous" or "in-context"

spectcai estimation.

AU linear dependence (mathematicai correlation) among the fkquency wrnponena

are accounted for in this simuitaneous estimate of the weighted least squares spect~m,

When the correlations between the frequency components are ignoreci, N homes a

diagonal manu of normal equation matrices Nk fm each individual frequencyfi and C& =

~ ~ - 1 . The expressions aven here are then equivalent to those in Sreeves [l98l], for the

independent estimation of spectral components where no decerministic mode1 is considered.

When the data are also equaily weighted, these expressions are identical to those in Vmiek

[1969a; 19721. When the data are equally spaced and the set of Touriei' frequencies are

used, N-1 = diag(2/n), and the weighted Ieast squares specnal values are then equivalent to

the standard one-side Fourier ones given by eqn. (3.4 1).

Vaniiek [1969a; 19721 also includes some simplifjhg trigonomerric identities that

make the evaluation of the elements in N-1 more efficient for equaliy spaced data (see ais0

Wells et al. [1983). These have been omitted h m the developments here for the sake of

simplicity, although any routine application of these shodd include these optimhations to

d u c e the required computationai effort

This approach is also similar to that used by Rochester et al. Cl9741 in that

correlations between different muencies are accounted for. However, the correlations

among the coefficients for same frequency are implicitly ignoreci in their expressions

because of the use of complex notation. The real (cosine) and imaginary (sine) tems for

the same kquency are treated independently. Only when the data are equaliy spaced is

theV approach equivalent to the preceding ones.

The same comments on the Fourier rransform regarding frequencies greater than the

Nyquist frequency also apply here for the sirnuitaneous estimate of the N l y weighted least

squares spectrum. Singularities in K.' should be avoided by using only frequencies up to

the Nyquist fkequency. Frequencies that are too closely spaced cm also cause ill-

conditionhg problems in the simuitaneous estimation of different spectral values.

Finally, it should be emphasized that these definitions of the least squares specmun

do not satisfy Parseval's relation. That is, the surn of the these spectraX values does not

equal the total power in eqn (5.7). Because of the correlation among the frequencies, there

is no equivalent to Parseval's relation for unequdiy spaced data.

5.5 Effect of Deterministic Mode1

In the developments thus far, the mathematical comlations mear dependence)

between the specnal components and any detemiùiistic mode1 have been ignored, as they

are in the traditional Fourier method. One of the most signincant contributions of VaniEek

[1969a; 1972 ] was the incorporation of the effect of any a priori deterministic mode1 in the

determination of the spectrai values. An important consequence (advantage) of this is that it

alleviates the need to deterrnine frequency response functions for the determùiistic d e l .

In the context of specmim estimation, fkquency response functions are used to account for

the effect of the detemiinistic model on the spectrum. Here, the determinisistic effects are

modeiled explicitly in the formation of the expressions for the estimation of the spectral

components.

The effect of the detemiinistic model on the spectrum is obtained in the same way as

for the inverse least squares trmsforrn in the previous chapter. The spectnun is defined as

the coniribution of each frequency component to the total power. This can be expressed in

terms of the quadratic fonn of the estimated Fourier ooefficients 2 as in eqn. (5.15).

However, to account for the effects of the deterministic model, the quadratic form must be

based on estirnates h m the combined detenninistic and "specaal" model as explained in

Section 4.5. That is, the spectral component sud for fiequencyfk is given by

where the ma& components for fkequency fk are, fkom eqn. (4.62).

Pk = N*-1 ** = N C I A ~ T p* @ ,

N*-1 and P* are dehed in eqns. (4.64) and (4.6 l), respectively. Note that these

expressions are formaily identical to those without a deterministic model. except that the

"reduced" weight ma& P* in eqn. (4.61) is used in place of P. The effect of the

deterministic model is thenfore completely containeci within p.

Foliowing the same substitution procedure as in the previous section, the Ieast

squares estimates of the spectral values cm be wrinen in terms of the weighted Ieast

squares msfotn FE in eqn. (4.67) as (cf. Vaniiek (1 97 1, eqn. (2.4)])

The normaiized spectrum is defined as before to be the percentage of the variation in

the data explaineci by the each spectral component. In the presence of an a priori

detenninistic model, this repments the variance explained by each spectraI component

which is not accounted for by the deterministic modeI. The part that is not explained by the

deterministic mode1 is just the residuals r~ finnom the deterrainistic mode1 alone. That is,

using the notation of Section 4.5,

where

and N m and UD are defined by eqns. (4.47) and (4.51), respectively. Expanding r D and

rearranging gives

a = ( z - A ~ N ~ ~ - ~ A ~ ~ P ) #.

Substituting this in the quadratic form of r~ and simpWyhg resuits in

rDTPrD = # T ( P - P A ~ N ~ ~ - ~ A ~ ~ P ) # = @'P*#, (5-26)

where P* is the "reduced" weight matrix accounting for the deterministic model. Dividing

the spectral values eqns. (5.22) by (5.26)- the n o d z e d spectnim that accounts for the

deterministic mode1 is

The consideration of which frequencies to include in the weighted least squares

spectmn must be done very carefully when accounting for the effects of a determuiistic

mode1 (it is effectively undefined in the spectruxn estimation). This is especidy important

if penodic trends are present in the d e t e d s t i c modeI. In that case, the spectral value for

the same frequency is undefined because it has effectively been accounted for in the

deterministic model and is therefore undefined in the least squares specmim; ie., the

periodic component in the detenninistic model and the same component in the spectrai

mode1 will be pesfectly lineariy dependent. Evaluating spectrai components for the same

fkequencies as the periodic mnds wiU mul t in a singular nomial equation manix p.

Present algorithms for the least squares spectrum (e.g., Wells et al. [1983) check for this

situation by inspecting the detemiiaant of Nk*; a zcro or near zero value indicates a

singdarity and thus an undefineci specnal value.

Ignoring oorrelations between spectrai components is @ d y acceptable within the

context of improvhg the deterministic modeL In this case the objective is to iteratively

search for only the largest spectral component in a residual &ta series h m a deterministic

modeL Any significant spectral values can then be incorporated into the deterniinistic

mode4 either explicitly as a periodic trend or irnplicitly as part of a more complex mudel of

the underlying physicai processes. In this way the rnethod effectively accounn for the

correlations among only the most sipnincant spectral cornponents that are iterativeiy

included in the dete-stic model.

5.6 Statistical Tests

Another p t advantage of the Ieast squares spectrum is that the signifïcance of the

least squares spccaal values can be tested statisticaiiy in a rigorous mamer. The following

statistical tests are based on S teeves [ 198 11.

It is weIl known in statistics that a quadratic fom has chi-square distribution with

degrees of W o m q u a i to the rank of the weight matrix. Expressing the estimated

spectral values in terms of the quadratic fom of the estimated Fourier coefficient& in

eqn. (5.15), this quantity then has a Chi-square distribution x ~ ( u ; 1-a) with u=2 degrees

of fkeedom (represen~g the rank of the covariance ma& C& for the two Fourier

coefficients for frequency fC) WaniZlek and Krakiwsky, 19863. A statistical test of the nuil

hypothesis %: SUR) = O can then be made using the decision function

S ~ 2 ( 2 ; L a ) ; accept Ho: su',) = O > ~ 2 ( 2 ; 1-a); reject Ho

where a is the signincance level of the test (usually 5%).

If the scale (Le., a priori variance factor 002) of Cg is unlaiown. the estimaM

value can be obtained fkom

where v = te2 is the degrees of freedom (two degrees of needom lost to the estimation of

the two Fourier coefficients). This estimateci variance factor is used to scale the covariance

ma& Ceb which then has a Fisher distribution F(v.u; 1-a) with -12-2 and u=2 degrees

of freedom. A statistical test of the null hypothesis I&: slfk) = O can then be made using

the decision function

{ I F(v ,2; 1-a); accept Ho: s(fk) = O > F(v.2; 1-a); reject Ho

The distribution of the normalized spectral values is obtained by h t rewriting the

quadratic foxm #T TP @ in temis of the residuals 3 and estimateci observations @hm the

spectral model. RealiPng that

the quadratic foxm of the residuals can be expressed as

r V r = # T ( P - P A N - ~ A T P ) ~ = $Tp*$.

Noting that P = Cg and rearranging,

Thus, the quadratic form of the observations is

which represent the total power. The quadratic f m on the right si& of eqn. (5.34) are

weLi known ( s e , e.g., Vaniiek and Krakiwsky 119861). The quadratic f m 9$' Cpk-I

Pk of the estimated Fourier coefficient has a Chi-square distribution with 2 degrees of

fkeedom (the number of Fourier coefficients for frequency fd. The quadratic form rT P r

of the residuals has a Chi-square distribution with w n - u degrees of freedom, whae u is

the total number of Fourier coefficients being simultaneously estimated (if the spectrai

values are king estirnateci independently, then u=2).

Using eqns. (5.15) and (5.34) in the expression for the nomalized spectrai value in

eqn. (5.1 8) and rearranging gives

where the ratio of ovo quadratic forms in the denorninator has the following Fisher

distribution

where "-4' means '7s distributed as", and v and 2 are the degrees of fieedorn of the

numerator and denominator, respectively. Note the use of the a probabilin/ level instead of

1-or This is because of the inverse relation between this F statistic and the spectral value

(for which we want the 1-a probabiiity level). Given the distribution of the ratio of the

quadratic fomis in eqh (5.35). the disaibution of the nomialized spectral value is then (cf.

S teeves 1 198 1, eqn. (3.19) 1)

A statistical test of the null hypothesis &,: = O c m then be made using the decision

fimction

The above Fisher disaibution can be simplifieci M e r using the inverse relation for

the Fisher distribution [Freund, 197 11,

When the first degree of hedom is NO, this can be approximated by [Steeves, 198 11,

This results in a statisticai test of the nuii hypothesis %: Svd = O using the decision

function

The statistid tests for specaal values that account for the presence of any

detemiùiistic mode1 are exactIy the same as above, except that the "reduced" observation

weight manix P* is used in place of the a d weight math P in the computation of the

quadratic forms.

The above tests are the so-cdied "out-of-context" tests, which test the individual

spectral components out of context of the others king escimated (see VaniZek and

Krakiwsky [1986, p. 229-23 11). They are identicai to those in Steeves [1981] and apply to

the independent estimation of the spectral cornponents, but not to the estimation of a i l the

specaal values simultaneously. In that case the "incontext" test should be used which

takes into consideration the estimation of the other spectral cornponents. Two approaches

can be used in this regard. The simplest one is to use the simultaneous confidence region

for all m fiequency components being estimatecl. This gives the same test as in eqn. (5.41)

except that 2m degrees of freedom is used in place of 2. However, this approach usuaily

results in tw pessimistic (large) a b i t to be any real value. A better approach is to use the

relation between the simultaneous probability a for the joint test of all spectrai components

together and the "local" probability a;, for the test of each spectral component separately.

Foilowing Miller 1 19661, the relation is given to first-order approximation by a, = dm.

The in-context test is then obtained by using in place of a in the above tests. NOIE that

Press and Rybicki [1989], Press et al. 11992, p. 5701 also use the in-context test based on

simultaneous probability. However. they incorrectly apply it to the testing of the

independently estiniated specnai components, where the correlations among the different

fkquency cornponents is ignoreci. The in-context test should only be used for the

simultanews estimates of the spectral values, where the correlations among al l the

frequencies used is accounted for.

5.7 Estimation Algorithms

As stated at the begidng of this chapter, th- have been a variety of papers since

Vaniiek, [1%9a, 19711 describing the same least squares spectrum (independently

estimated spectral components) in slightly different fom; e.g., Lomb 119751, Fenaz-

Mello 119811, Scargle [1982], Home and Baliunas [1986]. It can be shown, however, that

under the same assumptions ail of these are identical to Vaniiek's more general approach.

The ciifferences are only the use of slightly different nomiatization methods and different

numencal methods for solving the normal equations.

In Vaniiek [1969a], the direct inversion of the 2x2 normal equation matrix is

optimized by using an analytical expression. in addition to king the fastest algorithm, it

also accounts for the presence of a priori deterministic models and includes various

trigonometric identities for greater efficiency, especially for equally spaced data Compared

to the FFT, however, the least squares aansfom and specmun are cornputationaiiy much

slower. Unfortunately, a diioct comparison of computational speed could not be made

because of the software useci. AU tests were perfofmed using the MATLAB software,

which has a built-in (compiled) FlT function optimisai for speed whereas the least squares

specmun algorithm was implemented as an extemal (interpreted) function. Because

extemal fûnctions execute much more slowly than built-in functions, no fair comparison

between the FiT and least squares algorithms couid be made in M A T M . Nevertheless,

when confionteci with unevenly spaced data, the least squares method is the only axrect

approach to use.

Lomb [1975] and Scargie, 19821 solve the n o d equations using an

orthogonalization (diagonakation) procedure based on tirne shifnng (a different time shift

is needed for each frequency). This approach is slower than the dgecl analytical solution of

Vmiek. It &O does not account for the prtsence of any a priori models, except for a

mean. Ferraz-Mello [1981] uses Gram-Schmidt orthogonalization to diagonalize the

normal equarions. Again, this procedures is slower than direct anaiytical inversion and

does not account for the presence of any a priori deterministic modek.

Recently, Ress and Rybicki [1989] have developed a novel approach to the fast

computation of a least squares spectnim. It is based on the concept of "'extirpolation" and

the use of the FFî. Basicdy, exthplation gives an equally spaced data series bat, when

interpolateci to the original times, gives back exactly the original data series. This is also

called reverse interpolation. The FFT is used to evaluate the evenly spaced (extirpolated)

sine and cosine sununations in the tirne-shifting algorithm of Lomb [1975]. The original

extirpolation algorithm used two complex FFïs. The more efficient algorithm uses the

same trïgonornetric identities us& by Vaniiek [1969a] to reduce the computations to only

one FFT. The biggest disadvantage of this method is that it's lirnited to only the set of

ccFourier" frequenties due to the use of the FFT. It is thus not possible to " m m in" on

significant peaks to better resolve the frequency. The FFI' also requires 2" data points,

which necessitates zero-padding the data senes. As for the other algorithms, the presence

of a priori detenninistic models cannot be accounted for. Finally, the extirpolation accuracy

depends on the "oversampling factor" used in the extirpolation to genaate many more data

points than the original data senes. Gream oversampling of the extirpolated series

provides better accuracy but results in more computations. In spite of the above

limitations, this algorithms works very weU and very fast (on the order of n logn, ins&

of n2).

Chapter 6 Stochastic Modelling of Observation Errors

6.1 Introduction

The weighted least squares estimation model aliows for the stochastic modehg of

residual errors through the use of a M y popdated covariance manix. This can be used to

account for those systematic effects that have not been modelIed explicitly

(detefministically) in the design rnatrix fa the least squares model. The problem with

using f d y popuiated covariance maaices in this rnanner is the diaculty in detamining the

covariance or correlations among the observations in an objective way.

There are a few rnethods that cari be used to d e m e the variance and covariance

each with their own advantages and drawbacks. One of the most popular of these are the

rnethods of analysis of variance and variance-covariance component estimation. The

"anaiysis of variance" (ANOVA) method (also called factur analysis in statistics) cm be

found in most standard texts on statistics. Geodetic applications of the technique are

describeci in detail by Kelly [1991] and in a series of articles by Wassef 11959; 1974;

19'761. Essentiaily the aim of the method is to divide the measurements into separate

groups (factors which contribute to the overall variation in the data) and to estimate the

variance cornponents for each. The difficulty in applying the method is in dennuig a

scheme of dividing the observations into separate groups which charactcrize some

behaviour of the systematic eEect king modeiied. Often, the factors describing the

systematic effect cannot be so discretely defined, rather they are often of a continuous

nature that precludes lumping them together into separate and distinct groups.

Variance-covariance component estimation, on the other hand, is based on

modelling deteminktically the residual variation in the measurernents. The variances and

covariances are expressed in ~ ~ T K U S of iinear modek relating these components to various

factors describing the systematic effect The coefficients (variance and covariance

components) in the variance-covariance model are estimate- together with the paramettas in

a least squares solution. The technique is describai in detail in Rao and Kleffe [1988] and

has been appiied to many geodetic problems (see, e.g., Grafârend et al. [1980], Grafarend

[1984], Chen et al. [1990]). It can be shown that the analysis of variance method is just a

special case of this more general approach [Chnanowski et al., 19941. The problem with

applying the methad is that the estimation of the variance-covariance mode1 coefficients

usually needs to be iterated which can result in biased estirnates of the variances and

covariances [Rao and Kleffe, 19881. This c m lead to negative variances, which is

unaccep table.

The approach taken here is m mode1 any residual systematic effects remaining after

accounting for a detemiinistic model, usuig autocorrelation (ACF) or autocovariance

(ACVF) functions derived h m a power spectral density function of the residuals. This

idea was first proposed for geodetic applications by Vaniiek and Craymer [1983a; 1983b1

and M e r developed by Craymer [ 19841. To accommodate unevenly spaced dam, a

general least squares transfm is developed to determine the nomialized power spectrurn.

The inverse transforrn is then used to convert this to an ACF which is converted to an

ACvF.

6.2 Direct Autocovariance Function Estimation

The autocovariance function of an equaily spaced data &es i(td can be estimateci

directly using the expressions given in Chapter 2. This gives the sample autocovariance

func tion

1 C(z,) = - n-m

where rn = zm/& is the so-called lag number and dr is the data series spacing. No= that, as

in eqn. (2.20), the summation is divided by twn rather than by n, in order to provide an

unbiased estimate of C(rJ. The biased estimate is obtained by dividing by n.

For uneqdy spaced data which are relatively homogeneously disaibuteci, an

averaging procedure cm be used. In this approach the unevenly spaced lags are àivided

into equaily spaced lag intervals or bins, similar to the way in which histograms are

constnicted. Ail lags within the lag inmal are summed together in (6.1) to give an

average autocovariance for the lag interval. This method gives a smthed estimate of the

autocovariance function. The problem is that if the data have large gaps, the lag intervals

may need to be relatively large, resulting in degraded resolution See VaniZek and Craymer

[1983a;b] and Craymer (19841 for more details of this technique.

6.3 Autocovariance Function Estimation via the Spectrum

The autocovariance function for an evenly spaced data senes can be most

conveniently derived fiom the power spectrai density function using the Fourier transfom.

As discussed in Section 3.4, the autocovariance function c m be expresseci as the Fourier

transform pair with the spec- and the autocomlation function R(t) as the transfomi pair

with the nomialized spectrum. These expressions in terrns of the spectnun are ofien used

as the basis for the efficient computation of autocovariance and autocomlation functions of

evenly spaced data using the FFï. It wiU also be used as the basis for deveioping

autocovariance functions for unevenly spaced data to provide objective a priori estirnates of

covariances and weights that account for residual systematic effects in least squares

modellhg.

As mentioned in Section 3.4, care must be exercisad to avoid any "wrap amund" or

"end" effects when compu~g the autocovariance or autocomlation function b m the

s p e m This is most easily achieved by simply paddîng the data series with zeros out to

double the length of the original Senes. Furthemore, this indirect estimation via the

spectrum pmvides the biased estimate of the autocovariance/autocorrelation function. As

recommended by Bendat and Pierso1[1971, pp. 3 12-3141 and Pnestiey [1981, pp. 323-

3241, this should be used in preference to the unbiased estimate because the biased one is a

positive definite function which generates a positive definite covariance matrix. The

unbiased ACF and ACvF are not positive definite and result in suigular covariance marrices

that are not suitable for generating weight matrices for least squares models.

6.4 Iteratively Reweighted Least Squares Estimation

The covariance ma& generated h m the autocovariance function is used to

smchastically model the residual enas in the detaministic least squares d e l . The basic

idea is to begin with some a priori estimate of the covariance ma&, usudy a diagonal

maaix of known variances. A les t squares solution is obtained for the deterministic mode1

and the observation residuals provide an estimate of the random observation emrs. The

autocorrelation funaion is determine. for these residuals in order to obtain a more realistic

estimate of the correlations among the random obsewations e m . This autocordation

function is then used together with the a priori variances to generate a new covariance

function for the observations which is included in a new lest squares solution for the

dete-stic model and new estimate of the residual observation enws. Anorher

autocorrelation function is then computed and the whole estimation process is repeated

(iterated) until the solution for the detemiinistic mode1 and covariance matrix converge to a

stable form This is referred to an iteratively reweighted least squares estimation and is

identical to the iterated MINQUE technique exœpt that a dekministic mode1 is used tfiere

to rnodel the variances and covariances (see Ra0 and Klcffe [1988]). The procedure is

illustrated schematicaily in Figure 6.1.

from A Prion Variances

Weighted Least Squares 1 Solution for Deterministic Model

1 611 ~ovariansmatrix using ( ACF and A Priori Variances

I

Figure 6.1: Iteratively reweighted least squares estimation process.

Chapter 7 Numerical Tests

7.1 Introduction

In this chapter, various n d c a l tests of the least squares transfm and spectnim

are given under a variety of d.erent situations. Throughou~ the following terminology

and notation is used:

'Tourier" frequencies

LST

ILST

LSS

Independent LSS/ILST

Simultaneous LSS/ILST

Unweighted LSS/ILST

ACF

Indirect ACF

Set of integer multiples of the fundamental fkequency

Least squares transfonn

Inverse least squares transfm

Least squares spectnim

Independent estimation of the LSS or EST £i-equency

components

Simultaneous estimation of the LSS or ILST fhquency

components

Estimation of U S or ILST using equally weighted

obsexvations (no weight ma& P used)

Estimation of LSS or LST using weighted observations

(weight matrix P used)

Autocorrelation function

Indirect estimation of the autocomlation function via the

EST or the LSS

The tests presented here are based on simulateci data using a pseudo-random

number generator for n o d y distributed obmation e r r ~ r s and unif'ormiy distributed,

unequally spaced times. Udess otherwise stated, these tests use a deterministic mode1

consisting of a periodic trend with @od 10 (frequency 0.1 Hz). AU computations were

performed using the MAT- numerical and graphical software system.

Tests were perfonned to ascertain the effects of the following on the LSS and

indirect estimation of the ACF:

random observation emrs

correlations among observations

random sampling (unequally spaced data)

fkequency selection

deterministic modd

non-stationary random emrs (random walk)

The effects on the LSS and ACF were determined by comparing the results to the known

theoretical form for both functions.

7.2 Effect of Random Observation Errors

To study the effect of random observation mors, three data series of 100 equally

spaced points were used Each was composed of a penodic trend of amplitude 1 and

p e n d 10, Le., frequency O. 1 Hz. The first series contained no observation mrs. The

second series contained normally distributed random emrs with a standard deviation of

1B. The third data series contained nomaliy distributed random emrs with a standard

deviation of 2/3. The three data series are ploaed in Figure 7.1.

The least squares spectra (for "Fouxïer" frequencies) of the three data stxies are

given in Figure 7.2. &th the independently and simdtaneously estimatecf spectral values

wiU be identical in these tests because the data are equally spaced, equdy weighted and the

set of "Fourier" frequencies is used. The effect of random observation erm on the LS

specmmi is to reduœ the magnitude of the kgest spectral peak, which in ai l cases is

correctly located at the kquency of the periodic trend The iarper the random errer, the

greater the reduction in the spectrai value for the significant peak. The magnitude of the

reduction in the peaks is equivalent to the inverse of the square of the signal to noise ratio

(ratio of amplitude of periodic signai to standard deviation of noise).

The direct estimates of the autocorrelation hinctions for the three &ta series are

given in Figure 7.3. These are unbiased estimates and were estimateci using eqns. (2.20)

and (2.12). The ACFs al l exhibit the expected coslie form. However, the functions all

display correlations larger than one at large lags, typicai of the unbiased form. As

explained in Section 3.4, this s o - d e d "wild" behaviour is the main reason the unbiased

estimate is not used.

The biased estimates of the autocorrelation functions are aven in Figures 7.4 to 7.6

for the three data senes, respectively. Both the direct estimate and the indirect estimate via

the invme LS transfomi of the LS specmim are given as weiï as the difference between the

nvo. The indirect estimates were denved following the procedure described in Section 6.3,

where zero-padding is used to avoid any "wrap arounb' effem (see Section 3.4). As

expected, a l l three ACFs exhibit the correct sinusoicial shape and tapeMg characteristic of

the biased estimate. However, there is a reduction in the mgninide of the wrrelation as the

random error increases. AIthough the differences between the direct and indyect estirnates

get larger in direct proportion to the magnitude of the random error, they are negligible for

ail three data series.

Periadic Trend (14.1) + Random Enor (&)

Periodic Trend (f=0.1) + Random Emr (s=0.66ô7)

Figure 7.1: Periodic time series of 100 equaily spaced points and period 10 (ftequency

0.1 hz) with no obsewation errors and with nonnally disaibuteci random e m (standard

deviations 1/3 and 2/3).

Nonnalized LS Spectnim - Random Enor = 0.3333 1 , l I I ! I 1 1 I l 1

Frequency (Hz)

Nomlized LS Spectnim - Random Error = 0.6667

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Frequency (Hz)

Figure 73: Least squares spectra of time series of 100 equaiiy spaced points and period

10 (frequency 0.1) with no observation emrs and with nomaily dismbuted random emrs

(standard deviations 1/3 and Z). The horizontal line indicates the 95% confidence limit

for statisticdy signiFicant specaal peaks.

Direct ACF (Unbiased) - Random Error = O

-2 1 1 I 1 1 I 1 1 I 1 I O 10 20 30 40 50 60 70 80 90 100

Time h g

Figure 7.3: Direct estimation of unbiased autocorrelation functions of tïme series of 100

equaIly spaced points and period 10 (fiequency 0.1) with no observation mors and with

n o d y distributed mdom emrs (standard deviations ID and 2B).

Direct ACF (Biased) - Random Enor = O

t O - C, m - 2 8 O

O 10 20 30 40 50 60 70 80 90 100 Time Lag

Indirect ACF (E3iased) - Random Enor = O

O 10 20 30 40 50 60 70 80 90 100 Time Lag

x IO-'^ Indirect-Direct ACF (Biased) - Random Enor = O 6 I I t 1 I I I I 1

O 10 20 30 40 50 60 70 80 90 100 Tirne Lag

Figure 7.4: Cornparison of direct and indirect (via LS spectrum) estimation of biased

autocorrelation functions of tirne senes of 100 equally spaced points and p e n d 10

(frequency O. 1) with no observation mors.

Direct ACF (Biased) - Random Emr = 0.3333 1 I I 1 I I 1 1 1 1 L

Indirect ACF (Biased) - Random Enor = 0.3333 1 4 I ! ! I ! I I 1 1

i 1 1 I f I 1 I 1 I O 10 20 30 40 50 60 70 80 90 100

Time Lag

x lo4 Indirect-Direct ACF (Eased) - Random Error = 0.3333 OQ I I I I I I I I

-8 1 I 1 1 1 I I 1 I I 1 O 10 20 30 40 50 60 70 80 90 100

Time Lag

Figure 7.5: Cornparison of direct and i n k t (via LS spectrum) estimation of biased

autocorrelation bctions of time series of 100 equally spaced points and pend 10

(frequency 0.1) with random observation emns (standard deviation ID).

Direct ACF (Biased) - Random Error = 0.6667

1 -

O 10 20 30 40 50 60 7 0 80 90 100 Time Lag

Indirect ACF (Biased) - Randorn Enor = 0.6667

c O - C

cTI - $ O

- 1 i i i i i t i i 1

O 10 20 30 40 50 60 70 80 90 100 lime Lag

Indirect-Direct ACF (ûiaçed) - Random Error = 0.6667

O 10 20 30 40 50 60 70 80 90 100 Time Lag

Figure 7.6: Cornparison of direct and indirect (via LS spectnim) estimation of biased

autocorrelation functions of tirne series of 100 equaily spaced points and period 10

(frequency O. 1) with random observation enors (standard deviation 2B).

7.3 Effect of Correlated Random Errors

To test the effect of correlations among the random observation erras, it is

necessary to generate a comlated set of em>rs E This can be accomplished by finding a

nansfomtion L of a set of uncorrelated random errors with diagomi covariance matrix

Cq, which, by the law of propagation of ermrs, gives a set of correlated random errors E

with the desired covariance manix CE, Le., for identically nomially distributeci random

errors (C4=Z),

The above decomposition (factorization) of a manix into another ma& times the transpose

of itself is known as Cholesky decomposition, where L is a lower triangular ma& called

the Cholesky niangle or square root pahlquist and Bjorck, 1974, p. 158; Golub and Van

Loan, 1983, pp. 88; Ress et al., 1992, pp. 891. Using the Cholesky aiangle, the

t r a n s f o d set of correlated random errors can then be obtained from

In the following tests, the periodic data from the previous section is used with a

standard deviation of 2/3. A M y popdated covariance rnatrix for the observations was

constructed k m the autocorrelation fiinction

where At = 9 - ti = 1. A plot of the the series and correlation function are given in Figure

7.7 using a standard deviation of 2/3.

Three different types of least squares spectrum were cornputeci for this data series:

(1) the unweighted independent estimste, (2) the weighted independent estimate:, and (3)

the wcighted simuitaneous estimate. The different spectra a i l provide good results, each

clearly identifying the periodic component correctiy at kquency 0.1 (see Figure 7.8).

AIthough the unweighted independent LS spectrum displays slightly iarger noise at the

lower frequencies than the other spectra, the noise is weii within the 95% confidence

interval. The weighted LS specaa pvide almost identical resula, although the peak at

fiequency O. 1 is slightly larger. These resula verQ the daim by Steeves LI98 11 that

correlations among the obsmations have iittle effect on the resulting spectra.

The direct and indirect (via the unweighted inverse LS transform of the unweighted

LS spectrum) estimates of the autocorrelation function are given in Fi- 7.9. The two

ACFs are identical and agree well with the expected form for the perïodic data set (see

Figure 7.6), although those here display slightly larger correlations at lower fiquencies

duc to the a priori correlation function. The weighted indirect ACFs are shown in Figure

7.10. Both exhibit the correct shape for the periodic signal, but that based on the

independently esthateci spectnim gives larger correlations than for the unweighted

estimates. On the other hand, the ACF based on the simultaneously estimated specaum

displays much smaller correlations and thus gives the poorest estimate of the ACF.

Another check on the estimation of the autocorrelation functions was perfonned by

compu~g the ACFs only for the correlated ermrs (the periodic signal was not included).

The ACFs should agree closely with the a priori one used in C O ~ S ~ ~ U C M ~ the comlated

emrs (see bonom plot of Figure 7.7). Figure 7.1 1 shows both the direct and indirect (via

the unweighted inverse LS transfonn of the unweighted LS specmim) estimates of the

biased autocorrelation function. Both are identical and agree well with the theoreticai

correlation fùnction in Figure 7.1. The departures b r n the mie ACF are due to the

limitations of the random number generatm. The indirect weighted estimates via the

inverse weighted LS transfomi of both the independently and simultaneously estimated LS

specna are given in Figure 7.12. Ali these ACFs display the same shape, except for the

weighted simuitaneous estimate which has slightly larger currelations.

O 10 20 30 40 50 60 70 80 90 100 Time

Conelated Random Ems

O 10 20 30 40 50 60 70 80 90 100 Time

Autocorrelation Fundion of Random Errors

01 i i I i i 1 i I I O 10 20 30 40 50 60 70 80 90 100

Time Lag

Figure 7.7: Periodic time senes of 100 equally spaced points with period 10 (hquency

0.1) and correlated random observation emrs (standard deviation 2/3).

Unweighted lndependent LS Spectrum (Nomalized)

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Frequew (Hz)

Weighted lndependent LS Spectrum (Nomalized)

Weighted Simuitaneous LS Spectrum (Nomalized)

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Frequency (Hz)

Figure 7.8: Unweighted and weighted LS spectra (both independent and simultaneous

estimation) for periodic rime series of 100 e q d y spaced points with period 10 (fîquency

0.1) and correlated random observation emrs (standard deviation 2/3).

Direct UnweigMed ACF (Biased)

O 10 20 30 40 50 60 70 80 90 100 Time Lag

Indirect ACF (Biased) via UnweigMed Independent LSS 1 I I ! 1 ! ! I 1 I I

-1 1 i i i i 1 i 1 1 1 O 10 20 30 40 50 60 70 80 90 100

Time Lag

Figure 7.9: Direct and unweighted indirect (via unweighted inverse transform of

unweighted LS spectrum) estimates of biased autocorrelation function for periodic M i e

senes of 100 equally spaced points with period 10 (fkquency 0.1) and conelated random

observation mors (standard deviation 2D).

Indirect ACF (Biased) via Weighted Independent LSS

C 9 C m - 2 8 O

O 10 20 30 40 50 60 70 80 90 100 lime Lag

Indirect ACF (Biased) via Weighted Simuitaneous LSS

-0.5 1 i I i i i I 1 I i ! O 10 20 30 40 50 60 70 80 90 100

lime Lag

Figure 7.10: Weighted indirect estimates of biased autocorrelation function via weighted

inverse LS msform of both independent and simultaneously estimated LS spectra for

periodic time series of 100 equaily spaced points with period 10 (fkquency 0.1) and

correlated random observation emrs (standard deviation 2/3).

Direct Unweighted ACF (Biased)

-0.5 1 1 I I 1 1 1 1 I 1 I O 10 20 30 40 50 60 70 80 90 100

lime Lag

Indirect ACF (Biased) via Unweighted l ndependent LSS

Figure 7.11: Direct and unweighted indirect (via unweighted inverse transfoxm of

unweighted LS specmim) estimates of biased autocorrelation function for time series of

100 equaliy spaced points with correlateci random observation emrs only (standard

Indirect ACF (Biased) via Weig hted f ndependent LSS

lime Lag

Indirect ACF (Biased) via Weig hted Simukaneous LSS -

b

i - -- O 10 20 30 40 50 60 70 80 90 100

Time Lag

Figure 7.12: Weighted indirect estimates of biased autocorrelation function via weighted

inverse LS transfomi of both independent and simultaneously esthated LS spec~a for time

series of 100 equally spaced points with correlateci mndom observation emrs only

(standard deviation 2/3).

7.4 Effect of Random Sampling

Random observation sampling d t s in an unqually spaced data series in which

case the conventional Fourier expressions are no longer valid This is the primary reason

for using the least squares transfomi and spectrê To test the effect of random sampiing on

the LS aansform and spectm, unequdy spaced periodic data series were constructed.

Different lengths of data series were used to examine the effect of the finiteness and

sparseness of the data. The unequaüy spaced tirne arguments were created using a pseude

random number generator with a distribution (see Press et al. [1991] for an

explanation of the unifonn distribution). Thnx uaequally spaced (emrless) data sets with

a peridc trend of period 10 (fkequency 0.1 Hz) were generated with 100.60 and 20

points (see Figure 7.13).

The spectra were computed independently for integer multiples of the fundamental

frequency (0.01 h a , up a, fkquency 0.5 hz. Because the Nyquist fiequency is undefined

for randody data spacing, the spectra were wmputed only up to an arbitrarily selected

frequency of 0.5 hz. The absence of a Nyquist nequency is iilustrated in Figure 7.14a,

which gives the spectra of the data series up to maximum fi-equencies of 0.5,6 and 25 hz.

There is no evidence of a minor image in these spectra that would indicate the presence of a

possible Nyquist frequency. Also, because of the large correlations between the fiquency

components, it is not possible to estimate the simultaneous inverse LS transfm due to ill-

conditioning. This will be investigated M e r in the next section.

The spectra for the three data series are given in Figure 7.14b. The effect of

unequal samphg on the independent LS specmim is negligible. The specnal component at

fkquency O. 1 is correctly Iocared with a norrdked spectrai value of 1. The correct

location of the spectral peak is also unaffectai by the finiteness or sparseness of the data

series. Even with only 20 points the LS spectrum is practically unchanged, except foi

greater noise in the s p e m m and a larger 95% confidence leveL

The indirect (biased) esthates of the autocorre1ation function via the independent

LS specmun are given in Figure 7.15 for the three data series Zenpadding was used

prior to cornputhg the spectmm to which the inv- LS transform was applied. Ail ACFs

display the cmect shape and tapering for the periodic signal in the data series. The effect

of the random sampling is to reduce the magnitude of maximum correlation for non-zero

lags (compare top plot in Figure 7.15 with Figure 7.4). The maximum correlation is about

half of the theoretical f 1 value for all plots; i.e., the magnitude does not change as a

function of the finiteness or sparseness of the &ta. The correct shape of the theoretical

ACF is dso preserved even with ody 20 points.

For cornparison, Figure 7.16 gives direct estimates of the autown-elation functions

computed for the same unequally spaced data series using the intexval averaging method

describecl by Vaniiek and Craymer [1983a; 1983bJ and Craymer [1984]. AU ACFs display

the same periodic component as the indirect estimates (overlay Figure 7.16 with Figure

7.15). However, the direct ACF for the 100 point series clearly does not follow the

expected tapered shape (compare with Figure 7.4). Instead, the correlations at both smaU

and large time lags are significantly aaenuated, whüe correlations at the middle lags are

equal to one. It appears more like a modulateci unbiased A C The other ACFs agree weil

with both the indirect estimates; they are closer in magnitude to the theoretical ACF

(compare with Figure 7.4).

Unequally Spaced (Unifomly Distd) Series - 100 Points

Unequaliy Spaced (Unifomdy Distd) Senes - 60 Points

O 10 20 30 40 50 60 70 80 90 100 Time

Unequally Spaced (Unifonnly Distd) Series - 20 Points

O 1 O 20 30 40 50 60 70 80 90 Time

Figure 7.13: Periodic time series of different lengths of randody spaced points

(uniformly distributeci) with period 10 (£kquency 0.1) and no random observation ermrs.

Unweighted lndpendent LS Spectnim (NomaCzed) - 100 Points

O 1 2 3 4 5 6 7 Frequency (Hz)

Unweighted lndpendent LS Spectrum (Normaiïzed) - 100 Points a

O O 5 10 15 20 25

Frequency (Hz)

Figure 7.14a: LS spectra (independentiy estimated fkquency components) up to

diEerent maximum hqueencies for periodic data series of unequally spaced points with

period 10 (frequency O. 1) and no random observation mors.

Unweighted lndpendent LS Spednim (Nomalized) - 100 Points

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 F r m J e w (Hz)

Unweighted lndpendent LS Spectnim (Normalized) - 60 Points

Unweighted lndpendent LS Spectrurn (Normalized) - 20 Points 1 , I 7 n 1 1 l I I 1 I 1

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Frequency (Hz)

Figure 7.14b: LS spectra (independently estimated frequency components) for different

lengths of periodic data series of unequdy spaced points with period 10 (fiequency 0.1)

and no random observation emrs.

Indirect ACF (Biased) via Unweighted lndependent LSS - 100 Points 1Q 1 I I 1 I I 1 1 I 1

-1 1 I I I 1 1 I 1 I I 1 O 10 20 30 40 50 60 70 80 90 100

Tirne Lag

Indirect ACF (Biasecl) via UnweigMed lndependent LSS - 60 Points 1q I ! I r 1 I 1 r 1 1

-- . O 10 20 30 40 50 60 70 80 9 0 100

Time Lag

Indirect ACF (Biased) via Unweighted lndependent LSS - 20 Points

C O - C Ca - L O O

O 10 20 30 40 50 60 70 80 90 100 Tirne Lag

Figure 7.15: Indirect estimates (via unweighted inverse LS transform of unweighted LS

spectnun) of biased autocorrelation functions for different lengdis of periodic data series of

unequally spaced points with perîod 10 (fkequency 0.1) and no random observation errors.

Direct ACF (Biased) - 1 O0 Points

Direct ACF (Biased) - 60 Points

O 10 20 30 40 50 60 70 80 90 100 Tirne Lag

Direct ACF (Biased) - 20 Points

Figure 7.16: Direct estimates (via intmal averaging) of biased autocomlation functions

for different lengths of periodic data series of unequally spaced points with period 10

(fiequency 0.1) and no random observation enors.

7.5 Effect of Frequency Selection

The effect of different selections of fiequencies for the simdtaneous LS specmmi

was dso examined. Note that frequency selection only affects the simuitmeous estirnation

of the spectral components. It has no &ect on the independently estimated LS spectxum

where each spectral component is treated out-of-context of the ohen (no correlations arise)

and any set of frequencies may be used to correctly locate the significant spectral peaks in a

data series, within the limitations of the sampling theareni (see Section 5.3). This

effectively provides a continuous specaum, although spectral leakage may &ect the resdt

The significant spectrai components can then be used in the indirect estimation of the ACF

via the simuitaneously estirnateci LS transfurm or in an improved detenrnnistic d e l .

On the other hanci, the selection of hquencies is of critical importance for the

sirnultaneously esrimateci LS s p e m In this case the correlations among the spectd

components must be carefdly considered, othawise ill-conditioning in the normal

equations for the simultaneous solution of al l specaal components can produce completely

wrong resuits. For example, consider the same data series used in the previous section

(top plot in Figure 7.13), containhg 100 unequaiiy spaced ( u n i f d y distributed) points

with a periodic trend of period 10 (fkquency 0.1 Hz) and no random errors. Using the

entire set of 50 'Fourier" frequencies in the simultaneous LS specmun, results in an ill-

conditioned solution. The resulting spectrum fails to detect the penodic trend at fiequency

0.1 hz even with no random emns present (see top plot in Figure 7.17).

The correiations among the frequencies can be reduced and the ill-mnditioning in

the spectrai transfimn removed by decreasing the fnquency sampling to only every other

kequency; i.e., 25 of the original set of 50 frequencies. Although the periodic component

is now visible in the simultaneous LS specûum, it is still relatively small and only just

statistically significant (see middle plot in Figure 7.17). This is improved M e r by iaking

every 5th fiequency so that only 10 of the original 50 fÎequencies are u s e d The spectral

peak at 0.1 is now highly signikant.

The same behaviour is aiso dispiayed by the indirect estimate of the autocorrelation

function. Note, however, that the original data series needs to be zero-padded to avoid

"wrap mund" eEects in the ACF. This doubiing of the series length results in a

fundamental hquency that is haIf of that for the original series and twice as many

muencies. This results in even more severe ill-conditionhg and a completely emneous

ACF where correlations are much m e r than 1 (see top plot in Figure 7.18). Decreasing

the hquency sampling to only 50 hquencies irnproves the ACF but there are still some

correlations greater than 1 (see middle plot of Figure 7.18). The situation is improved

when only 10 frequencies are used. The ACF has the correct cosine fom and the

umùmum correlations are oniy slightiy la~ger than 1 (they couici be truncated to 1 in

pradce).

The problem with decreasing the fkquency sampling is that some peaks rnay be

missed. Clearly, great care must be exercwd when selecting the frequencies to use with

the simultaneous estimation of the LS spectmm and the inverse LS transfom. Note that by

reducing the number of simdtaneously estimated fhquencies, one is approaching the

method of independent estimation of the spectrai components (the extrerne or limiting case

of reducing the number of frequencies).

A better approach may be to instead search for and use only statistically signincant

spectrai components h m the uidependent estimation of the LS specnum. These

fkquencies can then be used in a simultaneous estimation of the LS spectrum and in the

simultaneous inverse LS transform for the indirect ACF. The results foilowing this

procedure are illusirated in Figures 7.19 and 7.20 for a randomly sampled data series with

two periodic components (fiequencies 0.1 and 0.25 hz) and no random mors. The

independent estimation of the LS specmim correctly identifies the two periodic cornponents

as shown in Figrire 7.19. Using only these significant periodic components in the

simultaneous estimation of the spectrum and the subsequent simuitaneous inverse

aansform gives an indirect ACF that agrees with the theoretical form of the unbiased. rather

than the biased, ACF, rathm than the biased as shown in Figure 7.20. On the other hanci,

the ACF deriveci b m the inverse ~ansform of the entire independently estimiited LS

spectnun provides the expected biased fomi Am. It appears that reducing the numbn of

fkquencies in the inverse transf' gives an ACF that more closely agrees with the

unbiased estimate. The biased ACF c m be obtained by simply using n in place of the

divisor (n-k) in the expression for the unbiased ACF in eqn. (2.20).

Weighted Simuttaneous LS Spednrm (Normaiized) - 50 Frequencies a

Weighted Simîtaneous LS Spedrum (Norrnafized) - 25 Frequencies

Frequency (Hz)

Weighted Simuitaneous LS Spectnim (Normalized) - 10 F r e q ~ e ~ e s 1 , 1 1 1 I I I 1 I I i

.................................................................. ;.................-

........ : ......... : ........ i i i i i i i i i i i i i i i i i i iJJJJJJJJJJ: --.....-

........ z ......... t . . . . . . . . : . . . . . r . . . . . . . . . . . . . . . . . . . . ,

.......... ........:......... I......... II..........................-

A r* A r* A b A. - v w V w Y

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 O S Frequency (Hz)

Figure 7.17: LS spectra for different sets of sirnulüuieously estimateci fnquencies for

periodic data senes of 100 unequally spaced points with period 10 (frequency O. 1) and no

random observation mrs.

Indirect ACF (Biased) via Unweighted Sirnuttaneous LSS - 100 Frequencies

O 10 20 30 40 50 60 70 80 90 100 Time Lag

-1.5 1 l I I I 1 I 1 I I I O 10 20 30 40 50 60 70 80 90 100

Tirne Lag

Figure 7.18: Indirectly estimated LS autocorrelation hctions via the LS spectrum using

different sets of simultaneously estkmted muencies for periodic data series of 100

unequaiiy spaced points with period 10 (frequency 0.1) and no random observation mors.

Unequaliy Spaced (Ultifomly Distd) Senes - 100 Points

O 10 20 30 40 50 60 70 80 90 100 Tirne

Independent LS Spectrum (Nonnalized) - 50 Frequencies 1 I I I I 1 I I I I I

Frequency (Hz)

Figure 7.19: Penodic time senes of randorniy spaced points with frequencies 0.1 and

0.25 hz and no random observation enors (top), and independent estimation of the LS

Indirect ACF via Independent LSS - 1 00 Frequencies

O 10 20 30 40 50 60 70 80 90 100 Tirne Lag

Indirect ACF via Sirnuitanmus LSS - 2 Frequencies

I .- - -- - - - -- -

O 10 20 30 40 50 6 0 70 80 90 100 Time h g

Figure 7.20: Indirecdy estimated ACF via the inverse LS transfomi of the independent

LS specmim using all frequencies (top) and of the simultaneous LS specaum using only

the two significant spectral peaks at 0.1 and 0.25 hz (bottom).

7.6 Effect of Deterministic Mode1

The ef fm of the determinisuc mode1 on the IS specmim and indirectly estimated

autocorrelation fiinction is to absorb any spectral components that are bighly correlatai with

the deterministic model. These spectral components are usually at the Iower fkquencies,

unless some high nequency periodic trends are included in the determimstic modeL The

deterzninistic mode1 is accomnrodated by accounting for its effect within the estimation of

the LS specrmm and inverse LS transfonn foilowing the appach described in Chapters 4

and 5,

To test the effect of a deterministic linear trend model, a 100 point equally spaced

data series consisting of a quadratic trend (1 + 0.02 t + 0.00005 t2) and a periodic residual

trend of frequency 0.01 hz was generated with no random enors (see top plot in Figure

7.21). The quadratic trend will tend to alias as a long period trend which may resdt in

erroneous estimates of the spectmm of the residuais if the correlations with the quacùatic

mode1 are not accounted for. This is evident in the rniddle plot of Figure 7-21, where the

LS spectcum displays a peak at 0.02 hz while the actual periodic signal should be at 0.01

h z There is also some spectral leakage into the neighbouring frequencies at 0.01 and 0.03

hz. Accounting for the correlations with the detemllnistic model results in a spectrum that

correctly identifies the 0.01 hz peak and eluninates the specaal leakage (bottom plot in

Figure 7.21).

lndependent LS Spectrum (Nomüzed) - Quadratic Trend Residuais

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 Frequency (Hz)

lndependent LS Spectnim (Nomlized) w/ Quadratic Trend

O 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 Frequency (Hz)

Figure 7.21: Quadratic trend t h e series with penodic component (fhquency 0.01 hz)

and no random enors (top); LS spectnmi of residuals from quachtic trend mode1 (middle);

LS spectmm accounting for effects of quadratic mode1 (bottom).

7.7 Effect of Non-Stationary Random Errors (Random Walk)

Another kind of correlateci error are non-stationary random enors. One example of

this is the simple random wak mode1 where the ermr 4 at tirne ri is the accumulation of a

white noise process [Papoulis, 19651; Le.,

where the Q are norrnally distributed random variables with zero mean. One such equaiiy

spaced random walk data sexies with a unit standard deviation is dispIayed in Figure 7.22

(top plot). This 100 point data series is actualiy a evenly sampled subset (every fifth point)

of a much larger 500 point random walk data series using a white noise process with unit

standard deviation. The theoretical spectnim for such a process is hversely proportional to

the square of the fkequency [Zhang et al., 1997. The computed LS spectrum is given in

the middle and bottom plots of Figure 7.22. The bottom plot uses a log scale for both axes

and exhibits a linear aend with a slope of about -2 correspondhg to the expectedf2

relation for a random walk rnodel. The direct and indirect autocorrelation functions are

aven in Figure 7.23. The indirect estimate via the LS spectnim (zero-padding is used)

agrees well with the direct estirnate. The differences between them shown in the bottom

plot of Figure 7.23 increase in direct proportion to the hg. The indirect ACF departs fimm

the direct ACF to about 0.5 at the highest lag.

To test the effect of the data sampling, an unevenly spaced mndom wak data series

was generated by randornly sampling the same 500 point random waik series used above

(see Figure 7.24). (A uniform random number genemor was again used to generate the

random selection of 100 points; see Section 7.4.) The LS spectruxn is given in the bomm

two plots. The effect of the random samphg is to 5tten out the specmun at the higher

mencies. The inverse square frequency relation only h01ds at the lower hquencies.

This behaviour was ais0 found by Zhang et al. [1997J. The indirect estimate of the

autocmlation function via the independent LS spectrum (with zempanriing) is also

si@cantly affected by the random samphg (see Figure 7.25). It now drops off much

more rapidly in cornparison to the direct estirnate in Figure 7.23).

Randorn Walk (Equaliy Spaced Times) - 1 0 Points 2 r 1 l I I I k I I

Independent LS Spectnirn (Normalized) - Random Walk

Frequency (Hz)

. . . . . - - . . . . S .

. . Y . . . . . . . .

Independent LS Spectrum (Norrnalized) - Randorn Walk

Log Frequency (Hz)

Figure 7.22: Evenly sampled 100 point random walk time series (standard deviation 1)

(top) and its corresponding LS specmm.

Direct ACF (Bi- - Random Walk

O 50 100 150 200 250 300 350 400 450 TÏme Lag

I&red ACF (Biased) via LS Spectrum - Random WaJk

Indirect-Direct ACF (Biased) - Random Walk 1 , I I I I I r I I I 1

O 50 100 150 200 250 300 350 400 450 Time Lag

Figure 7.23: Direct (top) and indirect (bottom) autocomlahon functions for 100 point

random walk data series.

Randam Walk (Unequally, Uniformfy Sampled Times) - 100 Points 2 , I I I I I I I 1 I I

lndependent LS Spectnirn (Normalized) - Random Walk

O 0.01 0.02 0.03 0.04 0-05 0.06 0.07 0.08 0.09 Frequency (Hz)

independent LS Spectmm (Nonnalized) - Randorn Walk .................................................. ........................... * . . . - . + . . - . . S . . * . * ........................................ -*..-*-...--...-~.--.-.-*.------..-- ....*...-.-.*.*.-. .............................................................................................. ......... .... r r ~ . . . - . * i . r r * . . r % r r b - i S - a r . I - r . t - - - r - - - - - a . . a... . . - 8 ...... .......... b . - . % . . b . - ~ . ....... ..........................................-.....--.......................... * * * . * * . . * * -

1 O-* Log Frequency (Hz)

Figure 724: Unevedy sampled 100 point random walk time series (top) and ia

corresponding LS spectrum.

Figure 7.25: Indirect estimate of autocomlation Ma the independentiy estimated LS

specsum for the uneveniy sampled 100 point random walk time series.

Chapter 8 Some Applications in Geodesy

8.1 Introduction

There have ken many applications of time series analysis in geodesy to the saidy

of tide gauge data, gravity data and geodynamics. In particular, the method of least squares

spectral analysis has been applied to studies of the Earth-pole wobble by Vanilek [1969b]

and Rochester et al. [1974]. However, there have been few applications of time series

anaiysis techniques to other h d s of geodetic data. The few studies employing these

techniques have been mostiy applied to levebg data (see, e.g., VanîEek and Craymer

[1983a, 1983b], Craymer [1984]. Vaniiek et al. [1985], Craymer [1985], Craymer and

VaniZek [1985, 1986,19901). More recently timc series analysis techniques have also

k e n applied to electronic distance measurement (EDM) data by Langbein et al. [1990] and

Langbein and Johnson [1997], and to Global Positioning S ystem (GPS) data by El-

Rabbany [1994], King et al. [1995] and Zhang et al. [lggq. In El-Rabbany [1994], only

standard Foinier (and FFT) methods in the equally spaced Mie dimension are considered.

The study by King et al. [1995] also assumed equally spaced time arguments. Only the

ment work of Langbein and Johnson [1997] and Zhang et al. [1997 have considered

unequally spaced dam in particuiar, Zhang et ai. [1997] have used the periodogram as

dehed by Scargle [1982], which can be shown to be a special case of Vmik ' s original

method (see Section 5.7). Estimation of covariance and correlation functions for stochastic

modelling of e m , however, was still based on traditional methods assuming equally

spaced data.

The sndies by Craymer et aL have applied time series techiques more generally to

arguments that are not necessarily equalîy spaccd in order to search for systematic e m ~ s

that depend on these quantities. Ail these studies have used the unweighted f m of the

independently estimated least squares spectrum to search for systematic errors in precise

levelling. Here, the weighted fonn of the least squares approach to specûum and

autocovariance function estimation are applied to the stochastic modehg of errors using

two mal examples: estimation of the defonnation of an EDM basehe across the San

Andreas fault using the same data as in Langbein and Johnson [1997J, and GPS single

point positioning using pseudo-range observations (the typical positioning data used by

most handheld GPS receivers).

8.2 EDM Deformation Measurements

Electronic distance measurements @DM') is the most precise distance measuring

technique at close to moderate ranges (about 1 km). The most accurate EDM instruments,

such as the Kem ME5000, cm routinely obtain submillimeter repeatability. The most

accurate EDM insrrument is based on dual kequency ("two-colour") lasers (see Slater and

Huggea [197q). The two measuring muencies d o w one to more directly detemine and

correct for the refraction effect (which is a function of the frequency of the laser). For this

reason, two-colour EDM insrruments are often used in southern California by Earth

Scientists to monitor the cnistal deformation mund the San Andreas fault (see. e-g.,

Savage and Lisowski [1995]).

Here the least squares specaal analysis technique is applied to the same data used

by Langbein and Johnson [1997] to search for possible systematic signals in their two-

colour EDM data. Traditional spectral techniques were used by Langbein and Johnson for

this purpose. Because the observations are at irregular time intervals, some necessary

approximations, specificaliy interpolation, had to be made to estimate their spectm. No

such approximations are needed for the least squares technique which is an ideal application

of this metfiod.

The &ta used in this analysis are part of the Pearblossom network. near Palmdale

in southern Cal i fha and were provideci by J. Langbein (personal communication, 21

February 1997) of the U.S. Geological Siirvey, Menlo Park, CA. The network is fadial in

design. whae aii distances (baseLines) are measured h m Holcomb to twelve smunding

monuments at distances h m 3 to 8 lan (see Figure 8.1). Ody the Holcomb-Lepage

baseline with a nominal distance of 6130 m was used in this andysis. Initially the baseline

measurements at Pearblossom were made s e v d times per week for 4 years (1980- 1984).

Since about 1987 they have been reduced to about once every 3 or 4 months, although each

baseline is measined twice during each network re-observation. In addition, different

instruments and monuments have been used over the years and thae have k e n a number

of earthquakes. Consequently, the data have been reduced to changes in basehe length

h m the nominai value and grouped into sets sharing common EDM instrumentation and

monuments between earthquakes. The time series of the Lepage baseline measmrnents is

given in Figure 8.2. Note the different offsets between each data group and the consistent

linear trend (expansion of the basehe) for a i l groups. The different danim offsets

represent biases in the measured differences due to the different instnment/monument

combinations or the occurrence of earthquakes. It was also noted that several observations

were repeated within a couple of hours of each other (two within 15 minutes!). To avoid

excessively large temporal correlations under these circumstances. only the second ( r e p t )

observations were used.

The *nt biases between measurement groups necessitate accounting for a

sepmate danim offset for each. Likewise, the consistent trend for aU groups necessitates

m o d e b g a common linear trend for a i l groups. Least squares estimates of these model

parametexs are given in Table 8.1, where the datum offsets are all refaenced to the b t

measurement epoch. The 1.72 i 0.07 mm&ear linea. trend (extension of the baseline)

Pearblossom network

F i 8.1: Location of the Pearblossom network in California used to rneasure crustai

deformation with a two-colour EDM instnunent and location of the Holcomb-Lepage

baseline spanning the San Andreas fault ninning through this network [after Langbein and

Johnson, 1997, Figure 11.

Holcomb-Lepage Baseline Length Changes 10, 1 I 1 I 1 I i l w h t

1 1 I I 1 1 1 I I 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998

Tirne (days)

Figure 8.2: Changes in length of Holcomb-Lepage baseline. Dinerent observation

groups are denoted by different symbol colowftype combinations.

Table 8.1: Least squares estimates of linear trend and d a m

offsets.

Estimate Std t Statistic

Offset #1 (mm) -2.3 O. 1 22.4

Offset #2 (mm) -3.2 0.2 17.0

Offset #3 (mm) -4.2 0.2 17.5

Offset #4 (mm) -4.8 0.5 9 .5

Offset #5 (mm) -15.4 0.4 35.0

Offset #6 (mm) -20.1 0.7 27.6

Offset #7 (mm) -7.1 0.4 20.3

offset #8 (mm) -10.5 0.5 19.5

Linear Trend (mm&) 1.72 0.05 34.4

agrees well with the 1.67 value determineci by Langbein and Johnson [1997]. In the l e s t

squares solution, the data were weighted using standard deviations provided by J.

Langbein (personal communication, 2 1 February 1997). AU estirnateci mode1 parameters

were statistically signifcant at any masonable significance level and were removed h m the

data leaving the residual series in Figure 8.3. It is this data mies that is used in the

following spectral analysis.

Before performing a spectral analy sis, appropriate frequencies (i.e., frequency

spacing and range) must be chosen. The total length of the data senes defines the smallest

frwluency spacing that cm be resolved without spectral "leakage" f h n adjacent peaks.

The freguency interval (4 is def'ined by

I I I I I 1 I I I 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998

Tirne (days)

Figure 83: Cornparison of residual baseline length changes after removal of estimated

distance offsets for each observation group and a common linear trend- Different

observation groups are denoted by different symbol colou~Itype combinations.

where T, = (?-min) is the fùndamental period andf, is the fundamental frequency (see

Section 3 .2 eqn. (3.17)). The largest frequency that can be detennined by the data series

is defined by the Nyquist frequency f ~ - It corresponds to the time interval over a triplet of

adjacent points, the minimum number of points for the unambiguous determination of a

periodic component

The Nyquist fkquency is not clearly defïned for unevenly spaced data. For evenly

spaced data, it is simply twice the time interval between any pair of adjacent points (Le.,

twice the sampling interval t). The Nyquist freq~ency is then defined as f~ = 2 /(2 t)

(cf. Section 3.2). This represents the largest fkquency (smdest period) the data series is

capable of reliably estimaihg without aliasing effects. For unevenly spaced data series, the

distribution of possible triplets of points can Vary si-cantly and thus there is no weU

d e f i Nyquist frequency present. In theory, the highest frequency (that can be estima&

h m a data series) will correspond to the srnaIlest point triplet interval. This intemil

corresponds to the srnailest pend (maximum fkquency) that can possibly be determined

£tom the data series. However, in practice, the specaa generally exhibit no mirror image

about this or any other frequency when the data are unevenly and r a n d d y spacd. The

exception is when deaüng with data that are reguiarly spaced as multip1es of some conmion

interval or evenly spaced except for gaps.

For the baseiine length residuais in Figure 8.3, the variation in possible Nyquist

frequencies is illustrami in Figure 8.4 in temis of histograms of the lengths (time intrrvds)

of all possible point triplets ("Nyquist periods")). The srnaIIest mplet interval is about 1 day

corresponding to a Nyquist frequency of 1 cylday. This is because the measurements were

collecteci on a regu1a.r daily basis in the beginning. In the following analyses, spectra are

therefore estimated at integer multiples of the fundamental fkquency up to a Nyquist

fiequency of 1 cy/day.

Histogram of Possible Nyquist Periods 40 I I I I I I I I I 1

O 2 4 6 8 1 O 12 14 16 18 20 Period of Point Triplets (days)

Histogram of Possible Nyquist Penods I 1 I I

O 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Period of Point Triplets (days)

Figure 8.4: Histograms of lengths of point mplets ("Nyquist periods") corresponding to

possible Nyquist muencies. Bottom plot gives a more detailed histogram at 1 day.

in the estimation of the weighted Ieast squares specaimi of the baseline lengrh

residuals, any linear dependence (mathematical correlation) with the estimateci detenninistic

mode1 (distances offsets and linear trend) are taken into account as described in Sections

4.5 and 5.5. The specmrm is plotteci in Figure 8.5 with respect to period instead of

frequency for d e r interpretation There are clear significant spectral components at

periods of 2 and 8 years, in addition to several peaks at perîods shorter than a year. The

lower plot in Figure 8.5 enlarges the short period range and shows significant spectrai

components at periods of about 1,100,150 and 200 days.

2 4 6 8 10 12 14 16 Period (years)

Weighted LS Spectrum of Residuals 1 I I I

.... : ......... : ................... : .......,

.... ;..................,.................,

....................... I.................,

y- -fl t

O 50 100 150 200 250 300 350 400 450 500 Period (days)

Figure 8.5 Weighted l e s t squares spectra (independently estimated) of baseline length

residuals h m the deterministic mdel in Table 8.1. The horizontal iine is the 95%

confidence interval for detecoing signincant spectral values.

The 8 year pexiod is interes~g because it is &O visible in the residuals between

about 1984 and 1996 (see Figure 8.3). It was thought that this might be due to a possible

additional datum offset at about 1988.7 in the data group between 1984.2 and 1992.5 (see

figure 8.2). Apparently, the instrumentation had been taken down and set up again at this

Mie but it was thought that this was done accurately so as not to produce any additional

bias in the distance measurements (J. Langbein, personal communication, 2 1 Mar& 1997).

To check for the significance of such a b i s , an additionai d a m offset was estimated at

l988.7. This resulted in replacing the 1984.2- l99Z.S group (with d a m offset #5) with

two oew groups; l984.2- 1988.7 with dahm offset #5 and 1988.7- 1992.5 with new datum

offset #5a Figure 8.6 shows these two new groups together with the time series of length

changes. The les t squares estimates of the mode1 with the additional offset (##a) are given

in Table 8.2 and the residual series after removing the mode1 is given in Figure 8.7. It was

found that the datum offsets #k5 and #a for the two new groups were statisticaily different

from each other at any reasonable significance level (t statistic = 7.0) and both biases were

therefore modelied in the foilowing analyses.

Holcomb-Lepage Basdine Length Changes 10, I I I 1 I I I 1 A l

Figure 8.6: Changes in length of Holcomb to Lepage basehe with additional d a m

offset in observation group from 1984 to mid-1992. Different observation groups are

indicated by different symbol colour/type combinations.

Table 8.2: Least squares estimates of Linear trend and datum

offsets, including additional datum offset (##a).

Estimate Std t Statistic

Offset #l (mm) -2-0 O. 1 19.3

Offset #2 (mm) -2.8 0.2 15.1

Offset #3 (mm) -3.6 0.2 15.5

Off't #4 (~nm) -4.2 0.5 8.7

Offset #5 (mm) -15.3 0.4 36-9

Offset H a (mm) -12.6 0.6 21.9

ûfF~et #6 (IMI) - 17.1 0.8 21.3

offset #7 (m~n) -6.2 0.4 17.4

offset #8 (mm) -9 -2 0.5 17 .O

Linear Trend (mmlyr) 1.51 0.06 26.9

Baseline Length Residuais (Weighted Sotn) 4 r 1 1 I I I I 1

Figure 8.7: Cornparison of residual baseline length changes after removd of estimateci

datum offsets, including additional offset, for each observation group and a common linear

trend for a i i groups. Different observation groups are denoted by different symbol

colodtype corn binations.

The weighted lest squares specmim for the rsiduals after removing the estimsted

detemnaistic mode1 with the additional damm offset is given in Figure 8.8. The most

obvious difference fiom the previous specmmi is that the peak at 8 years has now been

signincantly reduced by the introduction of the additional d a m offset in the model.

However, there still remains a large peak at about 1000 days (2.5 years) that accounts for

15% of the noise in the residual data series. One possible explanation for such an

interannual behaviour may be an El Nino wamiing efkct, wbich has hquencies of

between 2 and 4 years during tbïs time p a i o d The warming effect is generally

Weighted LS Spectrum of Residuals

O 2 4 6 8 10 12 14 16 Pe riod (years)

Figure 8.8 Weighted least squares spectra of basehe length residuals from the

detenninistic model with adnitional distance offset- The horizontal line is the 95%

confidence interval for d e t e c ~ g significant specfral values.

accompanied by more muen t and severe wet weather which couid cause monument

motion due to higher levels of ground water. In addition, the "piling up" of warmer waters

in the eastern Pacific could also possibly led to additional crusta1 loading on the westeni

seaboard of North America, The o h significant peaks are at short @ods and are more

clearly identifieci in the Iowa plot of Figure 8.8. The largest peaks in this fiequency range

a~ at about 150 and 210 days. Ciniously, these peaks are symmetrical (Hl days) about

small central peak widi a semi-annuai period (180 days). According to Vaniek [1969b],

this conesponds to a possible modulation of a semi-annual perïod by a 30 day period. The

semi-annual period may be related to weather. For example. it is well knom that southem

California generally has wet spring and fall and a dry summer and winter which could

conceivably cause a semi-annual period in the presence of gruund water, thus possibly

conaibuting to a semi-annual behaviour of the motions of the geodetic monuments. The 30

day p e n d may be related to lunar tidal effects. Other peaks evident in the spectrum are at

penod of about 110 days and 1 day. The diumal period is believed to be a consequence of

the usual diunial behaviour of many systematic effects related to atmospheric conditions,

such as atmospheric refraction and heating (expansion) of the ground and monuments. The

other notable feature of the s p e c m is the absence of an annuai period. In fact, the

spectral value for this pexiod is almost exactly zero, indicating that such a period had

already been removed h m the data. This was denied by Langbein (personal

communication, 2 1 March 2 997), however.

Langbein and Johnson [1997] also argue for the presence of a random wak signal

in the residual data series. Their specmim for the Holwmb-Lepage baseline was cornputeci

by first interpolaring the unevedy spaced measurement series to an evenly spaced one by

averaging the data spanning 15-35 days either side of the missing point White noise was

also added to their interpolateci value. The power s p e c m was then mmputed using the

FFT technique and plotted against the log of the frequency (see Langbein and Johnson

[1997, Figure 31). Their plots display a clear trend proportionai to 1/f2 as expecrad for a

Weighted LS Spectnim of Residuals 0 -2 . . . . , , , . . . . m . , 1 I ...

Weighted LS Spectnim of Residuais 1 oO I i I I

Figure 8.9 Semi-log (top) and log (bottom) plots of weighted least squares spectra of

basehe length residuals h m the detenninistic mode1 with additional datum offset. The

straight line represents a 4.60 linear trend at low frequencies (fc4x10-2).

random walk process (see Section 7.7). For cornparison, the weighted least squares

spectra is displayed in Figure 8.9 (top plot) using the same semi-log fkquency plor No

clear l/f* trend is apparent in this spectrum. The specmirn is also displayed in Figure 8.9

(bottom plot) using a fuii log plot, where the presence of random walk noise should

produce a negative hear mnd at low fi=equencies, as discussed in Section 7.7. A srnail

negative trend (-0.60 f 0.08) is visible in the Ieast squares spectnun at muencies below 4

x IO-* cy/day, which grows even smaller for higher frequencies. Howeva, this linear

trend is pmp0rtiona.I to l/fl-6, rather than 1 ~ 2 as characteristic of a random waik process.

The autocomlation fûnction for the observations was indirectly estimateâ nOm the

inverse Ieast squares transfomi of the independently estimateci, weighted least squares

spectrum following the iterative procedure outiined in Section 6.4. The a priori standard

deviations of the dam were used to generate a pria5 observation weights.. nie data series

was also zem-padded prior to computing the specûum to avoid any wrap around effects in

the autocorreIation function as described in Section 3.4. The main difliculty encountered

was with the large number of possible time lags for which the a u ~ m l a t i o n needed to be

computed. For uneveniy and randody spaced data, th= are in general as many different

lags as there are combinations of observation pairs. For the Holcomb-Lepage distance

measurements, there are 361 observations for which there are 65,341 unique possible time

lags (number of off-diagonal elements in the observation covariance matrix). It was

therefore impractical to compute the autocorrelation function for all lags at once. instead,

the ACF was computed separately for the lags corresponding to each row of the

observation covariance matrix. Oniy the autocorrelations for the upper triangular part of

each row needed to be computed. The entire correlation ma& R for the observations was

assembled in this way and the N1 covariance rnatrix C was obtained using the a prion

standard deviations of the observations (which were also used in the computation of the

weighted spectmm); i.e.,

C = S R S ,

where S is a diagonal matrix of the a priori standard deviations. The autocorrelation

function for is ploaed in Figure 8.10 together with an enlargement at short lags. Although

there is a periodic behaviour in the enlarged plot, the magnitude of the corre1ations are small

even for short lags. No explanarion was found for smaU correlation "spikes".

Autocorrelation Function

Figure 8.10: Indirect ACF, and enlargement at short lags, estimated from zero-padded

time senes of Holcomb-Lepage length changes with additional dam offset.

The detenninistic mode1 of the damm offsets and linear trend were re-solved using

the new full covariance matrix. The solution is given in Table 8.3 with the additional

datum offset (Ha) at 1988.7 included. Because of the smaU codations, there is little

difference in the estimated offsets and trend berneen this solution and that based on only a

diagonal covariance matrix (Table 8.2); ail are statisticaüy compatible. However, in most

cases the estimated standard deviations of the offsets and trend are larger when the full

covariance matrix is used, indicating greater uncertainty in the estimated parameters.

Table 8.3: h t squares estimates of linear trend and d a m

ofEseu, including additional offset (##a) and using estimated fidl

observation covariance matRx based on computed ACF.

Estimae Std t Sraastic

Offiet #1 O LI^) -1.7 0.2 7.8

offset #2 (mm) -3.6 0.2 16.2

Offset #3 (mm) -4.6 0.2 20.8

Off't #4 (III.~) -4.1 1.0 3.9

Offset #5 (mm) - 14.0 0.3 51.3

offset #a (mm) -11.9 1 .O 12.3

Offset #6 (mm) - 16.2 1.3 12.4

Offset #7 (mm) -5.8 0.6 9.3

offset #8 (mm) -8.9 1.2 7.4

Linear Trend (mm/yr) 1.44 0.09 16.3

Specifically, the standard deviation for the linear trend is increased h m 0.06 to 0.09

d y r . This is thought to be caused by a slight reduction in the overall redundancy due to

the linear dependence (mathematical corrdations) among the obsewations. There were also

some significant differences in the correlations between the estimated parameters. For

example, the codation between offsets #5 and #5a was reduced from 0.75 to 0.44. This

caused the difference between the two offsets to becorne Iess statisticaily significant (t

statistic reduced h m 7.0 to 2.4). Nevertheless, the difference is sall statisticaiiy

significant at the 95% confidence level, leading us to still consider the possibility that the

addition daturn offset is reai.

Table 8.4: Summary of estirnated linear trends with and without

extra offset and correIations.

Linear Trend k Standard Deviation

Without Corr. With Con-.

Without extra offset 1.72 k 0.05 1.61 k 0.07

1.67 m g b e i n & Johnson]

With extra offset 1.51 f 0.06 1.44 f 0-09

Fioaly, the estimateci hear trends (baseline expansion) are sunnnaiized in Table

8.4. The main ciifference with the estimate h m Langbein and Johnson [19m is due to

the use of the snnitional danun offset Ma. When the offset is not used, the estimiited trend

with or without the observation correlations is not signincantly different h m Langbein and

Johnson's. The Merences are weil within the 95% confidence intervals. When the extra

offset is used, the linear trends are reduced by about 0.2 mm/y with or without the use of

cmelations. These are sisnificantly different at the 95% confidence level The standard

deviation of the iinear trend is only slightiy increased by the additional offset

The use of observation correlations derived h m the esthateci autocorrelation

fùnction dso reduces the magnitude of the hear trends both with and without the extra

offset. However, the reduction is only about 0.1 in both cases and is not

statistically signincant at the 9596 conMence level. The correlations also increase! the

estimated formal standard deviations of the linear trends by about 5046, even though the

magnitude of the autocorrelation is relatively small. This increase is thought to be due to a

implied reduction in the total redundancy (the existence of wrrelations means the= are

effectively fewa truly independent observations).

Finally, it is noted that the estirriiited linear trend (1.44 f 0.09 mm&) with the extra

offset and correlations agrees beaer with the linear trend (1.46 mm/yr) estimated by

Langbein and Johnson for the basehe from station Holcumb to station Bird, which is in

the same generai vicinity and direction as station Lepage (see Figure 8.1). The baselines to

these two station shouid thedore behave similar1y in t e m of its motion relative to

Holwmb. The apparent agreement therefore suppom the existence of an extra d a m

offset in the measurements m Lepage.

8.3 GPS Point Positioning

The use of the Global Position S ystem (GPS) has grown greatly in recent years,

largely owing to the wide availability of d, low cost receivers. For an in depth

expianation of the concepts involved in GPS, see Wells et al. [198Q or Dana [1997]. In its

most basic mode of operation, referred to as the Standard Position Service, users can

obtain their position to an accuracy of only about 100 metres horizontally and about 150

metres venically. in this mode, GPS receivers make use of the secalled U A code pseudo-

range (measured satellite-to-receiver distance), which is obtained by timing the satellite-to-

receiver travel t h e of the basic U A (coarse acquisition) code that is superimposed on the

L1 carrier frequency. The satellite-to-receiver ranges are used to solve for the receiver's

position in what is essentially known as a 3-dimensional resection problem in siateying.

This mode of positioning is called "point positioning" to distinguish it b m other, more

accurate, methods based on relative or differentiai positioning between receiven.

Although the pseudo-range observable is capable of providing point positioning

accuracies of about 10 to 30 meaes, the US Depanment of Defense intentionaüy degrades

the observable to the 100 m level for seclaity reasons. This degradation is cded Selective

Availability (S/A) and involves the introduction of systematic enas in the fom of a

mathematical aigorithm (caUed cc~thering") into the hadcast satellite ephemais and clock.

To date only clock dithering has apparently k e n applied. This error propagates dVectly

into the signal travel time from which the pseudo-range observable is de-rived. However,

because the S/A error is fairiy systematic, there exist very large autocorrelations in the

pseudo-range data and thus also in the estimated positions derived h m them Hem, only

the vertical position is examined with the aim of investigating the degree of autocorrelation

in the &ta and the effect of using the autocdation fwiction to weight the point positions

when using tirne averaging to reduce the effects of S/A and improve the accuracy. The

malysis of the two horizontal componenu can be done in an analogous fashion.

The data used in this study were provided by W. Rescott of the US Geological

Siwey @asonal communication, 19 May 1994). They were obtained h m a geodetic

quality Ashtech L-XII GPS receiver and uicluded, for each masurement epoch, the receive

time, computed WGS-84 Cartesian coordinates of the receiver's antenna and computed

receiver clock b i s . The tirne series of instantaneous point positions refers to station

Chabot in the south part of Oakland, Ciùifoxnia. The point positions were recmded every

30 seconds for a total of 24 hours on April6, 1994. Plots of the variation in the horizontal

and vertical position estimates over this 24 hour period are given in Figure 8.1 1, and for

only the first hour in Figure 8.12. The high degree of autocomlation at short time intervais

is rcadily apparent h m the very systematic way in which the positions slowly Vary.

As already stated, the most cornmon method of reducing the effects of S/A is to

average the point positions over time. Generally, users average their positions over

intervals as short as 5 minutes and at most about an hour. Here, one hour averaging is

used to examine the effectiveness of this in reducing the effects of SIA. This provides for

24 independent hourly means.

For each hou, the least squares spectrum is computed. Any linear dependence

between the estimated mean and the specaal components is accounted for as describeci in

Section 5.5. The hour long subsets are also zero-padded to avoid any wrap around effects

in the derived autocomlation functions (see Section 3.4). The systematic nature of S/A is

revealed as statistically signifiant peaks in the spectra, mainly at lower frequencies. The

independently estimated least squares specaum for first hour of the height series is given in

Van'ation in Horizontal GPS Positions about Mean I

I I

-50 O 50 Change in East (m)

Variation in Vertical GPS Positions about Mean

O 5 1 O 15 20 25 Tirne (hr)

Figure 8.11: Variations in derived horizontal (top) and vertical (bottom) GPS positions

over 24 hours at station Chabot. Variation is with respect to mean position.

Variation in Horizontal GPS Positions about 24 hr Mean (Hour 1)

Variation in Vertical GPS Positions about 24 hr Mean (Hour 1)

1 O 20 30 40 50 60 Time (min)

Figure 8.12: Variations in recorded horizontal (top) and vertical (bottom) GPS positions

for the fim hou. at station Chabot Variations are with respect to 24 hour mean position.

Figrrre 8.13, where the SIA effects are ckarly evident. The peak around O. 1 c y / d (period

= 10 min) is typical also of the other hourly data sets. The spectra are sufficiently dinerent

h m hour to hour, however, so that it is not possibie to predict the S/A algorithm with ody

one hour of data.

The inverse least squares transform of the zero-padded spectrum was used to obtain

the (biased) autocorrelation function (ACF) for each hour. The biased estimate of the ACF

for the first horn is given Figure 8.14. There are smng, periodic corre1ations present

which are typical &O of the other hourly subsets. For each hourly subset the estimated

autocomlation function is used to populate a fidl wrrelation rnah fm the observations.

These are then used as the a priori weight matrix in the re-estimation of the hourly means;

referred to here as the "weighted" rneans. Note, however, that the ACF used in the

weighted mean was computed by k t removing the unweighted mean (using the identity

matrix for the a priori weight matrix). Consequently, the weighted mean was re-estimateci

by iterating the computation of the ACF using the weighted mean h m the previous

iteration as describeci in Section 6.4 and illustrated in Figure 6.1. The standard deviations

of the means were also scaled by the square mot of the estirnated variance factor of the

height residuais h m each hourly mean solution.

The un weigh ted means (equail y weighted observations with no comlations) and

weighted means (with comlations b a s d on the estirnated ACF) were obtained in the above

manner for each hourly subset of data. The results are s b z e d in Table 8.5 and plotted

in Figure 8.15. It is evident from these resuits that there is a systernatic vanation in the

hourly means. The average of the formal standard deviations is only about 4 rn whereas

the RMS of the hourly means is about 13 a This clearly indicates that the foxmal standard

deviations of the estirnated means are too optimistic. The use of observation correlations

derived h m the ACFs estimated fhm each hour of data does not -ove the results. The

average formai standard deviation is stU much smaller than the RMS.

Least Squares Spectrum w/ Zero-Padding (Hour 1)

Figure 8.13: Independently estirnated lem squares spectrum of GPS height

measurements for the first hour (data zen>-padded).

Biased Autocorrelation Function (Hour 1 ) I I I I 1 1

-0.5 I I I 1 I l O 1 O 20 30 40 50 60

Time Lag (min)

Figure 8.14: Indirect estimate of the biased autocorrelation function via the inverse least

squares transfm of the least squares spectnim for the first hour.

Table 8.5: Unweighted and weighted hourly means and their sondard deviations (Std) of

GPS height m«isurements over a 24 hour period. "A~erage"~ 'Xange" and "RMS" denote

the arithrnetic average (unweighted). range (max - min) and roo< mean square (about the

"Average"), respectively, of these hourly values and are provide for ülusaation only. - -

Unweighted Weighted Mean Std Mean Std

Hour 1

Hour 2 Hour 3 Hour 4

Hour 5 Hom 6

Hour 7

Hour 8 Hour 9

Hour 10

Hom I l Hour 12 Hour 13

Hour 14

Hour 15

Hour 16 H o u 17

Hour 18

Hour 19

Hour 20 Hour 21

Hour 22 Hour 23 Hour 24 - .-- - 244.776 - - - 241.927 9.138

Average 225.173 4.186 226,861 5.152

Range 47.519 3.874 52.466 8.455 RMS 12.576 13.691

1801 1 t I r 1 O 5 1 O 15 20 25

Hour

Unweighted Means

Figure 8.15: Unweighted (top) and weighted (bottom) hourly means of GPS height

280

g-260 Y

9 2 4 0 - H r 220 s = 200

180

measurements over a 24 hour period The ACF-weighted means are based on iterative

1 I I 1

- - - -

- -

- -

I I I 1

estimation of the ACF. Enor bars represent 95% confidence intervals.

O 5 10 15 20 25 Hour

h m these results it is clear that the one hour averaging does not eIiminate di the

effects of S/A. To gain a better understanding of the nature of S/A for this data set, the

least squares specaum was computed for the entire 24 hour data ser The specmun is

plotted in Figure 8.16 and the largest peaks are tabulated in Table 8.6. It is clear from these

results that there are many significant periodic components at periods longer than one hour.

in fact, the largest peak has a pend of about 1.5 hrs. There is also a peak at 24 hrs. which

is the fwidamental period of the &ta Senes. This wuld indicate a 24 hour mod (possibly

diunial atmospheric effects) or spectrai leakage h m periodic components beyond 24

hours. Without a longer data sexies, it is not possible to distinguish between these two

possibilities. Regafdless, it is evident that the hourly means are not capable of "averaghg

out" all of the systematic effects. Longer averaging is clearly needed

The hourly means were recomputed using the autoc0rreIation funaion h m the

entire 24 hour data set The autocorrelation function is plotted in Figure 8.17 and the new

weighted hourly means tabulated in Table 8.7 and plotted in Figure 8.18. The lower plot in

Figure 8.18 shows the Merences between these means and the unweighted ones. All

ciifferences are within the 95% confidence interval. Although the esrirnated means stili

exhibit the same systematic trend as for the unweighted ones, the foxmai standard

deviations are much larger. The average of these fornial standard deviations (12 m) is very

nearly the same as the RMS of the means (13 m). This indicates that the foxmal standard

deviations are now much more redistic, accounting for the uncertainty due to the systematic

effects still present in the resulrs.

As stated in the previous section, this increase in the formal standard deviations is

thought to be caused by a reduction in the o v d redundancy due to correlations m n g the

individual position estimates. The observations can no longer be considerd independent

of each other. The degrees of fieedom is thus somewhat less than the number of

observations minus the number of unknowns. The larger standard deviations represent a

Least Squares Spedrum

O 500 1000 Period (min)

Least Squares Spednim

O Period (min)

Figure 8.16: Least squares spectnim (independently estimated) for entire 24 hour data

set. The lower plot is an enlargement of the short period range. The horizontal line is the

95% confidence interval for d e t e c ~ g significant spectral values.

Table 8.6: Twenty of the largest peaks in Ieast squares spectnim in Figure 8.16.

Period (min) Specb.al Value

84.7 0.046

62.6 0.020

160.0 0.0 16

27.7 0.016

12.3 0.0 16 1440.0 0.016

41.1 0.015

36.0 0.0 14 72.0 0.0 13 16.0 0.0 13

13.7 0.0 13

32.0 0.01 1

15.2 0.01 f

144.0 0.01 1

19.2 0.0 1 1

22.9 0.01 1

25.3 0.01 1

17.3 0.01 1 16.2 0.0 1 1

28.8 0.0 10

Biased Autocorrelation Function (24 hrs) 1 I I

500 1 O00 rime Lag (dn)

Figure 8.17: Autocorrelation function for entire 24 hour data set

Table 8.7: Weighted hoinly means of GPS height measurernents and rheir standard

deviations (Std) over a 24 hour period using comlations from ACF base- on 24 hours of

data "Average", "Range" and "RMS" denote the arithmetic average (unweighted), range

(max - min) and root mean square (about the "Average"), respectively, of these hourly

values and are provide for illustration only.

Hour 1 Hour 2 Hour 3 Hour 4 Hour 5 Hour 6 Hour 7 Hour 8 Hour 9 Hour 10 Hour 11 Hour 12 Hour 13 Hour 14 Hour 15 Hom 16 Hour 17 Hour 18 Hour 19 Hour 20 Hour 21 Hour 22 Hour 23 Hour 24 249.42 1 21.283

Average 228.094 1 1.877 Range 45-9 17 15.769 RMS 13 ,289

ACF-Weig hted Means

O 5 1 O 15 20 25 Subset

Figure 8.18: Weighted (top) hourly means of GPS height measuremena over a 24 hour

period using correlations obtained h m ACF based on 24 hours of data, and clifference

with equaliy weighted means without comlations (bottom). Ermr bars represent 95%

confidence intervals. Note different vertical scale in top plot from that used in Figure 8.15.

more realistic estimate of precision than that deaived h m the unweighted m a n s without

any correlations among the individual height observations.

It should be r d h d that the S/A algorithm is very complex and a closely guarded

secret It cannot be assumed the effet would repeat f h n &y to &y to enable its

prediction. Furthemore, the systematic signals evident in the spectrum and autocorrelation

function may also be due to other effects, such as aopospheric and ionospheric delays and

signal multipath. Consequently, the systematic positionhg efzors can be expected to Vary

spatidy so that, ui general, an autocorrelation function computed for one point would not

be applicable for any other point Further investigations of qxctra for longer time periods,

at different times of the year and at diffant locations wouid be needed to make more

general conclusions about these effects.

Chapter 9 Conclusions and Recommendations

A general least squares (LS) transform and its inverse have been developed which

c m accommodate unequally spaced data, rancbm observation tl~uz?i, arbiuary fiequencies,

arbitrarily weighted and comlated observations, as well as the effect of any a priori

deterministic mode1 on the spectrai components. The L!3 specaum has aiso been

reformulated in terms of this LS tranSfm and its inverse. The conventional Fourier

transfom and derived spectrum have been shown to be just a special case of the more

general LS transform and spectrtun.

The LS transfomi avoids rnany of the limitations associated with the traditional FFT

algorithms. These inciude the requiremena that the &ta be equally spaced and have 2" data

points (some FFT algurithms exist for handling other than 2" data points, but these begin to

lose the computational efficiency of the m. It is also not limited to providing

information only about the 'Fourier" frequencies (Le., the integer multiples of the

fundamentai fkquency). On the other hanci, the LS transfom is much more

computationally demanding than the FFT.

The spectral cornponents in the LS spectrum and inverse LS tmnsform can be

estimated either independently (out of context) of each other or sïmuitaneously (in-wntext)

with each other. However, the simultaneous estimation is very sensitive, in tenns of U-

conditioning, to the choice of muencies while the independent estimation is not.

Independent estimation is thus a more robust estirnatm in general and was also found to

provide a more reliable estimate of the autocorrelation function. Nevertheless, it would be

possible to use the independent estimation for the identification of signifiant spectral

components, and to model only these components in the simulmeous estimatioa of the

inverse LS tfansfomi (for LS approximation) or in the LS specaum and the indirectly

derived a u m l a t i o n function. This would result in a more optimal estimation of the

inverse wnsform and the specmmi, where the wmlations among the estimated specnal

components are taken hto account Care must be taken, however, when selecting these

components to avoid any iuconditioning in the sirnultanaus estimation due to

multicollinearity among the components (see Draper and Smith [l98 1, p. 2581 for a

discussion of multicollinearity). In such cases, o d y the independently estirnated specaal

components can be used. It is recommended to fiirther investigate the advantages and

disadvantages of SimultaneouIy estimating the spectral components.

The most signiFicant application of the LS transfm is in the rigomus determination

of autocomlation functions for unevenly s p d data series. Autocorrelation functions c m

be indirectly denved h m the inverse LS transform of the LS spectrum. These c m then be

used to construct M y populared a priori weight manices for more complete rnodelling of

systematic effects in least squares estimation problems. The tests with real data show that

the use of fully populated weight matrices generally results in an increase in the magnitude

of the standard deviations of the estimateci model parameters. This provides more realistic

estimates of the uncertainties and is thought to be caused by an implicit reduction in

redundancy due to the correlations arnong the observations.

The efiect of correlations among the observations on the estimation of the LS

spectnun was found not to be very significant, as claimed by Steeves [Mi 11, even when

the correlations are relatively large. This is an important consideration because ~ C C O U ~ M ~

for the correlations requires the use of a fuily populated observation covariance ma& in

the weighted f o m of the LS expressions derived here. which imposes a great

computational burden. Ignoring the comlations was found to have littie effect on the

results (estimated specrra and model patameters) but it significantly increases the

computational efficiency. On the other han& as stated above, the use of a priori

correlations among the observations generally gives more realistic

uncertainties associateci with the estGnated mode1 parameters

generally resuits in overly optimistic estimates of the standard deviatons.

Although the algorithm for the LS transform and specmim is relatively slow

compared to the FFI; the computationai efficiency can be greatly impved by using the

trigonometnc identities employed by VaniEek [1971]. It is strongly recommended to use

these in any practical use of the LS tmnsform and specaum. It is also recommended to

investigate the use of "windowing" techniques widely used in traditional Fourier spectra to

reduce the effecn of aliasing (see, e.g.. Press et al. [1992, pp.545-55 11).

Finally, the direct and inverse LS transfomis have been developed here oniy for

one-dimensional problems. Thus, it is srrongly reconmiended to investigate the possibility

of extending the LS transform to multiple dimensions in order to dow for the estimation of

multidimensional spectra and autocorrelation functions. This is especially impairtant in

geodesy, where most problems are of a two- or three-dimensionai name and where

observations are affected by a wide variety of (multidimensional) systematic emrs. In

these cases, a multidimensional form of the LS transforms should ideaily be used.

References

Bendat, J.S. and A.G. Piersoi (197 1). Radom Duta: Anolysis and Measwernem

Procedues. John Wiley and Sons, Inc., New York.

Bracewell, R.N. (1989). The Fourier transform. Scientific Amencan, Vol. 262, No. 6,

June, pp. 86-95.

Bracewell, RN. (1986). The Hartley Traifonn. Oxford University Press, London.

Castle, RO., T.D. Gilmore, R.K. Mark, B.W. Brown, Jr. and R.C. Wilson (1983). An

examination of the southern Calrfomia field test for the systematic accumulation of

the optical refhciion e m r in geodetic leveling. Geophysicd Resemch Letters, Vol.

10, NO. 11, pp. 1081-1084.

Chen, Y.C. (1983). Analysis of deformation srirveys - A generaiized approach. Technical

Report No. 94, Depamnent of Geodesy and Gwmatics Engineering, University of

New Brunswick, Fredericton, N.B.

Chen, Y.C., A. Chrzanowski and M. Kavouras (1990). Assessrnent of observations

using unbiased minimum nom quadratic unbiased estimation (MINQUE). CISM

Journal, Vol. 44, No. 4, pp. 39-46.

Chrzanowski, A., M. Caissy, J. Grodecki, J. Secord (1994). Software development and

naining for geomeaical defornation analysis. Contract Report 95-00 1, Geodetic

Slwey Division, Geomatics Canada, Ottawa.

Cooley, J.W. and J. W. Tukey (1965). An algorithm for the machine calculahon of

complex Fourier series. Mathematical Computing, Vol. 19, pp. 297-30 1.

Craymer, M.R. (1984). Data senes analysis and systematic effects in levelling. M.A.Sc.

Thesis, Dept. of Civil Engineering (Survey Science), University of Toronto,

Toronto; Technïcai Report No. 6, Survey Science, University of Tmnto (Erindale

Campus), Mississauga, Ontario.

Craymer, MM. and P. Van%ek (1985). An investigation of systematic ermrs in Canadian

levebg lines. Roc. Third International Symposium on the North American

Vertical D a m , Rockville, MD, A p d 21-26, pp. 441-447.

Crayrner, MM. and P. VaniCek (1986). FuRher analysis of the 1981 Southern California

field test for levelling refi-action J o d of Geophyscul Research, Vol. 9 1, No.

B9, pp. 9045-9055.

Craymer, MR and P. Vaniiek (1990). A cornparison of various dgorithms for the

spectral analysis of unevenly spaced data series. Presented at the Joint Annual

Meeting of the Canadian Inshtute of Sweying and Mapping and Canadian

Geophysical Union, Ottawa, May 22-25.

Craymer, M.R., P. VanEek and R.O. Castle (1995). Estimation of rod scale enors in

geodetic leveling. Journal of Geophysical Research, Vol. 100, No. B8, pp.

15129- 15145.

Dahlquist, G. and A. Bjorck ( 1 974). Nwnerical Methoh. Pren tice-Hall, Inc., Englewood

Cliffs, NJ.

Dana, PH. (1997). Global Positioning System Overview. The Geographer's M t

Project, Department of Geography, University of Texas at Austin. World Wide

Web address <h ttp://www.utexas.ed~depts/grg!gcraft/notes/gps/gps.h tmb

Deeming, T.J. (1975). Fourier analysis with unequally-spaced data Astrophysics and

Space Science, Vol. 36, pp. 137- 158.

Draper, N.R. and H. Smith (198 1). Applied Regression Analysis. Second Edition. John

Wiley & Sons, New York.

El-Rabbany, A. and A. Kleusberg (1992). Physical correlations in GPS differential

positioning. Ebc. Sixth International Symposium on Satellite Positioning,

Columbus, OH, March 17-20.

El-Rabbany, AE-S. (1994). The Effkct of Physical Comlations on the Ambiguity

ResoIution and Acciiracy Estimation in GPS Differential Positioning. Technical

Rtport No. 170, Department of Geodesy and Geomatics Engineering, University

of New Brunswick, Fredericton, N.B.

Fesraz-Mello, S. (1981). Estimation of paiods from unequdy spaced observations. The

Asnommicul JournclZ, Vol. 86, No. 4, pp. 619-624.

Freund, J E (1 97 1). MuthematicaZ Statistics. Second Edition. Prentice-Hall, Inc .,

Englewood W s , NJ.

Golub, G.H. and C.F. Van Loan (1983). M d Cmpuratiom. The John Hopkins

University Press, Baltimore, MD.

Grafarend, E.W. (1976). Geodetic applications of stochastic processes. Physics of the

Earth und PImerary Interiors, Vol. 12, pp. 15 1 - 179.

Grafarend, E.W. (1984). Variance-mvariancecomponent estimation of Heimert type in

the Gauss-Helmert model. ZfV, Vol. 109, pp. 34-44.

Grafarend, E.W., A. Kleusberg and B. Sch& (1980). An introduction to the variance-

covariance component estimation of Helmert type. ZfV, Vol. 105, pp. 16 1- 180.

Hartiey, R.VL. (1942). A more symmetricd Fourier analysis applied to transmission

problems. Roceedings of the IRE, Vol. 30, p. 144.

Holdahl, S.R. (1981). A rnodel of temperature stratification for the correction of levehg

refraction. Bulletin Geodesique, Vol. 55, No. 3, pp. 231-249.

Home, J.H. and SL. Baliunas (1986). A prescription of p e n d aoalysis of unevenly

sampled time series. The Astrophysicol Journal9 Vol. 302, pp. 757-763.

Kac, M. (1983). What is random? American Scientist, Vol. 7 1, pp. 405-406.

Kelly, K.M. (1991). Weight estimation in geodetic leveiiing uskg variance components

derived from analysis of variance. M.A.Sc. Thesis, Dept of Civil Engineering

(Survey Science), University of Toronto, Toronto.

Kreyszig (1978). Introduciory F~uu:tioruaI Andysis with Applications. John Wiley and

Sons, New York,

Langbein, J. and H. Johnson (1997). Conelated emns in geodetic t h e series:

hglications for time-dependent deformation. Joumal of Geophysical Resemch,

Vol. 102, NO. B 1, pp. 59 1-603.

Langbein, J.O., R.O. Burford and L.E. Slater (1990). Variations in Fault Slip and Snain

Accumulation at PaMield, California.. Initial Results Using TweColor GeOdimeter

Measurements, 1984-1988. J o m l ~Ge~physica l Research, Vol. 95, No. B3,

pp. 2533-2552.

Lomb, N.R. (1976). Least-squares frequency analysis of unequally spaced data.

Asrophysics and Space Science, Vol. 39, pp. 447-462.

Lucht, H. (1972). Korrelation in prazisionsniveiiement 'Wissenschafliche Arbeiten der

Lehrstule fur Geodasïe, P hotogrammetrie und Kmograp hie, Nr. 48, Technisc he

Universitat Hannover.

Lucht, H. (1 983). Neighbourhood-conelarion among observations in levelling networks.

In H. Pelzer and W. Niemeier (Editors), Precise belLing : Con~butions to the

Workrhop on Precise k e l l h g , Dümmler Verlag, Bonn, pp. 315-326.

Maul, G.A. and A. Yanaway (1978). Deep sea tides determination h m GEOS-3. NASA

Contract Repon 141435, National Aeronautics and Space Administration, Waiiops

Flight Center, Wallops Island, VA.

m e r , R.G. (1966). Simultaneolcs Statistical inference. McGraw-Hill, New York.

Moritz, H. Covariance functions in least-squares coilocation. Report No. 240,

Department of Geodetic Science, The Ohio State University, Columbus, OH.

ONeiU, M.A. (1988). Faster than fast Fourier. Byte, Vol. 13, No. 4, pp. 293-300.

Papoulis, A. (1965). Probabiliry, R a n h Vanobles. and Stz~chastic Processes. 2nd

Edition. McGraw-Hill Book Company, New York.

Ress, WH. and G.B. Rybicki (1989). Fast aïgorithm for spectral analysis of unevenly

sampled data. The Amophysicul Jozunai, Vol. 338, pp. 277-280.

Press, WH. and S.A. Teukolsky (1988). Search algorithm for weak periodic signais in

uneveniy spaced data. Conputers in Physàcs, Vol. 2, No. 6, pp. 77-82.

Press, WH., BP. Flannery, S.A. Teukolslq and W.T. Venerling (1986). Numerical

Recipes: The Art of Scienrific Computing. Cambridge University Press,

Camùxidge.

Press, WH., S.A. Teukolsky, W.T. Vetterling and B.P. Flannery (1992). Numerical

Recipes in FORTRAN: The Art of Scienrifc Computing, Second Edition.

Cambridge University Press, Cambridge.

Priestley, M.B. (198 1). Spectral Anaiysis and Tàme Series. Academic Press, London.

Rao, C.R. and I. Kleffe (1988). Estimation of Vuriance Components and Applications.

North-Holland, New York.

Rochester, M.R., O.G. Jensen and D.E. Smylie (1974). A Search for the Earth's 'Nearly

Diurnai Free Wobble'. Geophysicd J o m l of the Royal Astronomkal Society,

Vol. 38, pp. 349-363.

Savage, J-C. and M. Lisowski (1995). Geodetic monitoring of the southern San Ancireas

fa* California, 1980- 199 1. Journal of Geophysicul Research, Vol. 100, pp.

81 85-8192.

Scargle, JD. (1982). Studies in astmnornical time series analysis II: Statistical aspects of

spectrai analysis of unevenly spaced data. The Asrrophysical Journal, Vol. 263,

pp. 835-853.

Schwarz, K.P., M. Sideris, E.G. Anderson, P. Stoliker and S.M. Nakiboglu (1986). A

weight estimation scheme for the NAVD 88 readjusment. Conaact Report 86-02,

Geodetic S urvey Division, Geomatics Canada, Ottawa.

Seeber, G. (1973). Data evaluation by covariance analysis, exercised on photographie

satellite observations. Proc. Symposium on Eanh's Gravitational Field and Secular

Variations in Position, pp. 454-462.

Sjoberg, LE. (1983). Unbiased estimation of variancecovariance components in

condition adjusmients with unknowns - A MINQUE approach. SfV. Vol. 108,

pp. 382-387.

Slater, LE. and G.R Huggett (1976). A multiwavelength distance-measuring insrnunent

for geophysical experiments. Journal of Geophysical Research, Vol. 8 1, pp.6299-

6304.

Steeves, R.R. (1981). A statistical test for significant peaks in the least squares specmun.

In Collected Papers, Geodetic Survey, Sweys and Mapping Branch, Energy,

Mines and Resources Canada, Ottawa, Ontario.

Stein, R.S. and W. Thatcher (1 982). Field test for refraction in leveling. Eos

Transactions, American GeophysicaZ Union, Vol. 63, No. 45, p. 1106.

Stein, R.S., C.T. Whalen, S.R. Holdahl, W.E. Strange and W. Thatcher. Saugus-

Palrndale, California, field test for reection in historical levelling surveys. Journal

of Geophysicai Research, Vol. 9 1, No. B9, pp. 9056-go??.

Taylor, J. and S. Hamilton (1972). Some tests of the Vmiek method of spectral analysis.

Astrophysics and Space Science, Vol. 17, pp. 357-367.

VaniEek, P. (1969a). Approdate specnal analysis by least-squares fit. Asnophysics and

Space Science, Vol. 4, pp. 387-391.

Vmiek, P. (1 969b). New Analysis of the Earth-Pole Wobble. Studia Geophysica et

Geodaetica, Vol. 13, pp. 225-230.

Vantiek, P. (1 97 1). Further developrnent and properties of the spectral analysis by least-

squares. Asnophysics and Space Science, Vol. 12, pp. 10-33.

Vaniiek, P. (1979). Tensor structure and the lest squares. Bulletin Geodesique, Vol. 53,

pp. 22 1-225.

Vaniiek, P. (1986). Diagpmmatic approach to adjusmient calculus. Geodezja, Vol. 86,

No. 999, pp. 29-39.

Vaniiek, P. and M.R. Craymer (1983a). Autocorrelation functions as a diagnostic tool in

leveihg. In H. P e k r and W. Niemeier (Editon), Precire LeveIIing:

Conaibutions ro the Workrhop on Precke Levelling, Diimniler Verlag, Bonn, pp.

327-341.

Vaniiek, P. and M.R. Craymer (1983 b). Autocorrelation hinctions in the search for

systematic errors in levelling. Mm~(scripfa Geodaercu, Vol. 8, pp. 32 1-341.

VamZek, P., GH. Carrera and UR Craymer (1985). Corrections for systematic emrs in

the Canadian levelling network. Technical Report No. 10, Survey Science,

University of Toronto (Exindale Campus), Mississauga, Ontario.

VaoiZek, P. and E.W. Grafarend (1980). On the weight estimation in levelhg. NOAA

Technical Report NOS 86 NGS 17, U.S. Department of C o m c e , National

Oceanic and Amspheric Administration, National Ocean Survey, Rockville, MD.

Wassef, A.M. (1959). Nore on the application of mathematical statistics to the analysis of

levelling errors. Bulletin Geodesique, Vol. 52, pp. 19-25.

Wassef, A.M. (1 974). On the searc h for reliable criteria of the accuracy of precise levefig

based on statis tical considerations of the discre pancies. Builetin Geodesiqw , Vol.

112, pp. 149-163.

Wassef, A.M. (1976). Propagation of enors in precise leveiiing and its bearing on the

assessrnent of ment crustal movements. Bulletin Geodesique, Vol. 53, pp. 53-60.

Weiis, D.E., N. Beck, D. Delikaraoglou, A. Kleusberg, E.J. Krakiwsky, G. Lachapelle,

R.B. Lmgley, M. Nakiboglu, K.-P. Schwarz, J.M. Tranquilla and P. VaniEek

( 1987). Guide to GPS Positioning. Canadian GPS Associates, Fredericton, N.B.

Wells, D.E., P. Vmiek and S. Pagiatakis (1985). Least squares spectral andysis

revisited. Technical Report No. 84, Department of Geodesy and Geomatics

Engineering, University of New Brunswick. Fredericton, N.B.

Whalen. C.T. and W.E. Strange (198 1). The 1981 Saugus to Palmdale, California

leveling refiaction test NOAA Technical Report NOS 98 NGS 27, National

Geodetic Information Center, Rockville, Md.

Zhang, J., Y. Bock, H. Johnson, P. Fang, J. Genrich, S. Williams. S. Wdowinski and J.

Behr (1997). Southen California Pemianent GPS Geodetic Anay: Emn Analysis

of Daily Position Estirnates and Site Velocities. Jownal of Geophysical Resemch.

in press.

l MAGE EVALUATION TEST TARGET (QA-3)