1
Estimation of Clock Parameters and Performance Benchmarks for
Synchronization in Wireless Sensor Networks
Qasim M. Chaudhari and Dr. Erchin Serpedin
Department of Electrical and Computer EngineeringTexas A&M University, College Station, TX.
2
Outline
1. Wireless sensor networks
2. Related work
3. Clock model
4. A Sender-Receiver protocol
5. Clock offset estimation
6. Clock offset and skew estimation
7. Simplified schemes
8. Best Linear Unbiased Estimation – Order Statistics
9. Minimum Variance Unbiased Estimation
10. Minimum Mean Square Error estimation
3
11. Clock synchronization of inactive nodes
12. Clock offset and skew estimation in a Receiver-Receiver protocol
13. Conclusions
14. Future research directions
4
Wireless Sensor Networks
Gateway
Internet
Server
Wireless Terminal
S
S
D
Source
Destination
D
S
5
Introduction
Small scale sensor nodes Limited power Harsh environmental conditions Communication failures Node failures Dynamic network topology Mobility of nodes
6
Applications Monitoring
Environment and habitat Military surveillance Security Traffic
Controlling and tracking Industrial processes Fire Detection Object tracking
7
Main Challenges
8
Importance of time synchronization
Time synchronization in WSNs is important for Efficient duty cycling Localization and location-based monitoring Data fusion Distributed beamforming and target tracking Security protocols Network scheduling and routing, TDMA
9
Constraints Limited hardware
Reduced computational power Low memory
Limited energy Communication vs. computation RF energy required to transmit 1 bit over 100 meters is equivalent to execution
of 3 million instructions [Pottie 00]
Traditional clock synchronization techniques Communication comes for free Computational resources are powerful
Examples: NTP is energy expensive, GPS is cost expensive
10
Related Work
Reference Broadcast Synchronization (RBS) [Elson 02] Conventional receiver-receiver protocol Reduces nondeterministic delays Conserves energy via post facto synchronization
Timing synch Protocol for Sensor Networks (TPSN) [Ganeriwal 03] Conventional sender-receiver protocol Two operational phases: Level Discovery and Synchronization
Time Diffusion Protocol (TDP) [Su 05] Achieves a network-wide equilibrium time using an iterative, weighted
averaging technique based on diffusion of timing messages
11
Related Work
Analysis of a sender-receiver model [Ghaffar 02] For known fixed delays, maximum likelihood estimator for clock offset
does not exist Five algorithms: median round delay, minimum round delay, minimum
link delay, median phase, average phase. Minimum link delay algorithm has the lowest variance
Maximum likelihood clock offset estimator for unknown fixed delays [Jeske 05]
12
Clock Model
A computer clock consists of two components Frequency source Means of accumulating timing events
Practical clocks are set with limited precision Frequency sources run at slightly different rates Frequency of a crystal oscillator varies due to
Initial manufacturing tolerance Temperature, pressure Aging
13
Clock Model
A general clock model can be represented by
where is the clock offset, is the clock skew and is the clock drift
Clock synchronization problem Given the logical clock for a node k in the network, then
is a function of Target synchronization accuracy Amount of energy the network is willing to pay
14
2,kT
4,kT
3,kT
1,kTA
B
2 1 4 3( ) ( )
2 2
T T T T U V
1 4
2 3
, : :
, : :
T T
T T
Local time stamps at Node A Fixed portion of delays
Local time stamps at Node B Clock offset
1. Node A sends a timing message (Level of Node A and T1) to Node B at T1.2. Node B sends an ACK (Level of Node B, T1, T2, and T3) to Node A at T3.
With this, Node A calculates the clock offset.
Sources of error (time uncertainty) associated with message exchanges
o Send time: time spent to construct a message
o Access time: delays at MAC layer before actual transmission
o Propagation time: time of flight from one node to another
o Receive time: time needed for the receiver to receive the message and process it
A Sender-Receiver Protocol
15
Observations
Fixed clock offset model is not sufficient in practice Clock skew correction results in long term synchronization
and hence more energy savings Network delays being asymmetric is a more realistic scenario Even for the symmetric clock offset only model, better
estimation schemes achieving are possible Minimum Variance Unbiased Estimation (MVUE) Minimum Mean Square Error Estimation (MMSE)
Lack of analytical performance bounds and metrics Average RBS error: [Elson 02] or [Ganeriwal 03]?
16
Clock Offset
Gaussian Noise Assumption One motivation comes from experimental basis [Elson 02] In case of unknown delay distribution, we can evoke Central
Limit theorem Example: for uniform delays, the sum of even two of them
starts resembling a Gaussian curve
17
Clock Offset
The likelihood function can be written as
And the clock offset estimate and the CRLB are
2,kT
4,kT
3,kT
1,kTA
B
18
Clock Offset
Exponential Delay Assumption Random delays often modeled as exponential Several traces of delay measurements on Internet collected by
[Moon 99] fitting an exponential distribution Conformation of experimental observations with mathematical
results Experimental observations
Minimum link delay algorithm [Paxson 98] Clock Filter algorithm in NTP [Mills 91]
Mathematical results Best performance by Minimum link delay algorithm [Ghaffar 02] ML estimate based on minimum order statistics [Jeske 05]
19
Clock Offset
Likelihood function is given as
ML clock offset estimate is
CRLB is derived as
20
Clock Offset and Skew
1,kT1,NT
2,1T
4, 1,1( 1)( )NT T
Node A
Node B
4,1T 1,2T 4,2T
3,1T 2,2T 3,2T
4,kT
2,kT 3,kT2,NT 3,NT
4,NT
: clock offset
: clock skew
1,kT
4,kT1,1 0T
1,( 1) k kT X
4,( 1) k kT Y
1,1T
kX kY
2, 1,( )k k kT T X
3, 4,( )k k kT T Y
21
Clock Offset and Skew Gaussian
Likelihood function with is
Joint ML estimate for clock offset is shown to be
where
22
Clock Offset and SkewGaussian
And for the clock skew
Computationally quite complex Fixed delay must be known Open problem: Recursive implementation/update?
23
Clock Offset and SkewGaussian
Cramer-Rao Lower Bound is expressed as
where
Proportional to clock skew squared Not only dependent on number of synchronization messages
but also on the synchronization period
24
Clock Offset and Skew Exponential
The likelihood function in this case is
Four different cases need to be considered
Case I Known Known
Case II Known Unknown
Case III Unknown Known
Case IV Unknown Unknown
25
Clock Offset and SkewExponential
Case I: known, known
Constraints
ML estimator
26
Clock Offset and SkewExponential
27
Clock Offset and SkewExponential
Case II: known, unknown
Constraints
Lemma 1: lies on one of
the following curves
28
Clock Offset and SkewExponential
Lemma 2: lies either on point A or to the left of it (B,C,…) Lemma 3: To the left of A,
boundary of support region is
formed by a sequence of
curves with decreasing slopes Lemma 4: is unique
and is given by one of
29
Clock Offset and SkewExponential
30
Clock Offset and SkewExponential
31
Clock Offset and SkewExponential
Case III: unknown, known
Constraints
32
Clock Offset and SkewExponential
Lemma 5: Only two points satisfy the constraints ML estimator has the closed-form expression
33
Clock Offset and SkewExponential
Case IV: unknown, unknown
Constraints
Curves intersect on the line
Over this line, is constrained by
34
Clock Offset and SkewExponential
Problem can be solved by the application of four lemmas Final form of the ML estimator is
35
Clock Offset and SkewExponential
36
Clock Offset and SkewExponential
37
Simplified Schemes
Fixed delay must be known in Gaussian case Computational complexity Further simplification within the same framework is possible
suitable for WSNs in case Synchronization accuracy constraints are not stringent Energy conservation constraints are strict
One simple scheme is independent of delay distribution involved
Cost paid is slight degradation in estimation quality
38
Utilizing Data Samples 1,N
Better skew estimation for large synchronization period Utilize only 1st and last sample differences for eliminating the
clock offset Define Simplified new model
where and are either Gaussian or Laplacian distributed depending on original delay distribution
39
Utilizing Data Samples 1,N
Gaussian delays Likelihood function for highly reduced data set is
ML-Like clock skew estimator is expressed as
CRLB-Like lower bound is Depends on timestamping “distances”
40
Utilizing Data Samples 1,N
Exponential delays The reduced likelihood function is
ML-Like clock skew estimator can be derived as
CRLB-Like lower bound
41
Utilizing Data Samples 1,N
Simulation results
42
Two Minimum Order Statistics
Motivation Unknown delay distribution Small synchronization period
Opening the model equations as
Choose two points as
43
Two Minimum Order Statistics
Joint the two points to obtain the estimate through its slope and intercept
The form of the estimator is
Almost as simple as the clock offset only case Knowledge of is not required
44
Two Minimum Order Statistics
45
Two Minimum Order Statistics
Simulations results
46
Two Minimum Order Statistics
Computational complexity comparison with the MLE
47
Summary
Gaussian Exponential
Offset Model MLE + CRLB CRLB
Offset + Skew Model MLE + CRLB MLE + Algorithms
Offset + Skew Model Using First and Last sample
ML-Like + LB
Using First and Last sample
ML-Like + LB
Offset + Skew Model Two minimum order statistics
Algorithm + Computational Complexity
48
Best Linear Unbiased Estimation – Order Statistics
Limited power resources in WSN implies better estimation techniques should be utilized
Results derived so far correspond to symmetric delays, although asymmetry is a more realistic scenario
Best Linear Unbiased Estimation (BLUE) is suboptimal in general due to linearity constraint
What if the linearity constraints are on the order statistics of observed data, instead of the raw observations?
49
Best Linear Unbiased Estimation – Order Statistics
Transforming the data as
Following relations hold for ordered data
50
Best Linear Unbiased Estimation – Order Statistics
The covariance matrix for can be derived as
Its inverse can be found by Gauss-Jordan elimination Let the ordered observations be represented as
51
Best Linear Unbiased Estimation – Order Statistics
Asymmetric Link Delays The asymmetric linear model can be written as
And the Gauss-Markov theorem implies
52
Best Linear Unbiased Estimation – Order Statistics
The covariance matrix for is
The final expression for is
53
Best Linear Unbiased Estimation – Order Statistics
Symmetric Link Delays The symmetric linear model can be written as
The Gauss-Markov theorem yields the solution
54
Best Linear Unbiased Estimation – Order Statistics
Covariance for is
The expression for is
BLUE-OS for same as the MLE for symmetric link delays
55
Minimum Variance Unbiased Estimation
Asymmetric Link Delays Found by the application of Rao-Blackwell-Lehmann-Scheffe
theorem Likelihood function can be expressed as
According to Neyman-Fisher factorization theorem, the sufficient statistics is
56
Minimum Variance Unbiased Estimation
Notice that Find such that
Applicable only if is a complete sufficient statistic
Only function of is unbiased
57
Minimum Variance Unbiased Estimation
Unbiased estimator of ? Note that BLUE-OS is unbiased and hence MVUE !
Compensation for asymmetry through
Variance of the clock offset
58
Minimum Variance Unbiased Estimation
Symmetric Link Delays Again applying the Rao-Blackwell-Lehmann-Scheffe theorem,
the likelihood function is
More than one unbiased functions of complete statistic? Through Neyman-Fisher factorization theorem, the actual
sufficient statistics is
59
Minimum Variance Unbiased Estimation is proved to be complete Unbiased estimator of ? BLUE-OS is unbiased and hence the MVUE
In symmetric case, the MVUE and BLUE-OS of coincide with MLE
Its variance is
60
Summary
Clock Offset
MVUE
Symmetric Delays MSE
Remarks
Asymmetric Delays MVUE
MSE
Remarks
61
Explanatory Remarks
Does this discontinuity in clock offset estimates performance make sense?
Which estimator is better when the network delays are slightly symmetric?
MVUE is the best in unbiased class of estimators, not all. For asymmetric case,
62
Explanatory Remarks
The MLE outperforms the MVUE under the condition
Estimator could be chosen according to the number of synchronization messages if knowledge of is available
Around the point , MLE attains lesser MSE as the link asymmetry decreases, i.e.,
63
Explanatory Remarks
Simulations results
64
Explanatory Remarks
Apparently, adapting between the two estimators a good idea according to
since have been obtained too. Despite the fact that MLE is functionally invariant,
considerable amplification of errors occurs due to repeated nonlinear processing
Results are even applicable to Internet time synchronization
65
Explanatory Remarks
As a byproduct, the MVUE of the fixed and mean variable link delays are obtained
Endd-to-end delay measurements are helpful in analyzing network performance
Very useful for applications behaving adaptively based on observed network performance
Continuous media applications, such as audio and video, absorb the delay jitter perceived at receiver for smooth playout of media stream
66
Minimum Mean Square Error Estimation
In general, the MMSE estimator is not realizable due to the dependence of MSE on the unknown parameter
MSE is a sum of variance and bias squared
This dependence usually comes from the bias Setting the MSE proportional to inverse of the scale parameter
cancels the dependent factors
67
Minimum Mean Square Error Estimation
The MMSE estimator comes out to be a function of MVUE Closed-form expression for
And for mean link delays
MSE of clock offset
68
Minimum Mean Square Error Estimation
The MMSE estimator comes out to be a function of MVUE Closed-form expression for
And for mean link delays
MSE of clock offset
69
Clock Synchronization for Inactive Nodes
Packet synchronization protocols Receiver-Receiver (R-R) Sender-Receiver (S-R)
For WSNs implementing any sender-receiver protocol, the inactive nodes can exploit the timing messages received
m p
qm p
q
70
Clock Synchronization for Inactive Nodes
The model can be represented as
m
p
q
1ms 1
p mr mjs p m
jr p mJr
mJs
1m pr
1p ms m p
jr p mjs m p
Jr p m
Js
1m qr
1p qr m q
jr p qjr m q
Jr p q
Jr
71
Clock Synchronization for Inactive Nodes
Likelihood function, assuming symmetric delays, is
The maximum likelihood estimator is derived as
72
Clock Synchronization for Inactive Nodes
73
Clock Synchronization for Inactive Nodes
The pdf of is obtained as
Cramer-Rao Lower Bound is
Is the ML estimator efficient?
74
Clock Synchronization for Inactive Nodes
An efficient estimator does not exist due to the rule “if an efficient estimator exists, the ML procedure will produce it”.
Simulation results
75
Clock Synchronization for Inactive Nodes
Symmetric delay assumption was less realistic Better estimation techniques can be employed Using the transformed data,
the linear model can be written as
76
Clock Synchronization for Inactive Nodes
and
Hence, the covariance matrix is
Final form of estimator
77
Clock Synchronization for Inactive Nodes
The likelihood function in symmetric case is
MVUE for the clock offset is derived and shown to coincide with BLUE-OS and MLE
Its variance is give by
78
Clock Synchronization for Inactive Nodes
Similarly, for asymmetric link delays, the BLUE-OS is
which is also the MVUE Its variance is given by
79
Clock Synchronization for Inactive Nodes
MMSE estimator can be derived as a function of MVUE Closed-form expression for
And for mean link delays
MSE of clock offset
80
Clock Offset and Skew in a Receiver-Receiver Protocol
Main sources of errors – send time and channel access time – are removed
A receiver-receiver model can be represented as
81
Clock Offset and Skew in a Receiver-Receiver Protocol
The likelihood function can be expressed as
where
Objective function to be maximized is
over the constraints
82
Clock Offset and Skew in a Receiver-Receiver Protocol
JML estimator
83
Clock Offset and Skew in a Receiver-Receiver Protocol Why Gibbs Sampler?
Joint ML estimator is biased MVUE does not exist since the sufficient statistics depend on
unknown parameters
Posterior distribution can be found and clock parameters can be estimated by its mean for better results
Straightforward extension to additional unknown parameters, e.g., clock drift
Posterior distribution involves complex integrations, hence the Markov-Chain Monte Carlo (MCMC) methods
84
Clock Offset and Skew in a Receiver-Receiver Protocol
Algorithm for Gibbs Sampling is to iterate the following with initial values :
After a threshold value , the set behaves as the sample values from the joint posterior
85
Clock Offset and Skew in a Receiver-Receiver Protocol
In the current scenario, the Gibbs Sampler is implemented as follows
Performs better than the JML estimator Simulated with
MVUE as the lower bound with one parameter known BLUE as the upper bound due to linearity constraints
86
Clock Offset and Skew in a Receiver-Receiver Protocol
Simulation results
87
Conclusions
General exponential family model Accumulated error analysis for multihop protocols Effect of mobility on time synchronization
88
Future Research Directions
General exponential family model Accumulated error analysis for multihop protocols Effect of mobility on time synchronization