1. 1 Estimation of Clock Parameters and Performance Benchmarks for Synchronization in Wireless Sensor Networks Qasim M. Chaudhari and Dr. Erchin Serpedin Department of Electrical and Computer Engineering Texas A&M University, College Station, TX.

2. 2 Outline 1. Wireless sensor networks 2. Related work 3. Clock model 4. A Sender-Receiver protocol 5. Clock offset estimation 6. Clock offset and skew estimation 7. Simplified schemes 8. Best Linear Unbiased Estimation – Order Statistics 9. Minimum Variance Unbiased Estimation 10. Minimum Mean Square Error estimation

3. 3 11. Clock synchronization of inactive nodes 12. Clock offset and skew estimation in a Receiver-Receiver protocol 13. Conclusions 14. Future research directions

4. 4 Wireless Sensor Networks Gateway Internet Server Wireless Terminal S S D Source Destination D S

5. 5 Introduction  Small scale sensor nodes  Limited power  Harsh environmental conditions  Communication failures  Node failures  Dynamic network topology  Mobility of nodes

6. 6 Applications  Monitoring  Environment and habitat  Military surveillance  Security  Traffic  Controlling and tracking  Industrial processes  Fire Detection  Object tracking

7. 7 Main Challenges

8. 8 Importance of time synchronization  Time synchronization in WSNs is important for  Efficient duty cycling  Localization and location-based monitoring  Data fusion  Distributed beamforming and target tracking  Security protocols  Network scheduling and routing, TDMA

9. 9 Constraints  Limited hardware  Reduced computational power  Low memory  Limited energy  Communication vs. computation  RF energy required to transmit 1 bit over 100 meters is equivalent to execution of 3 million instructions [Pottie 00]  Traditional clock synchronization techniques  Communication comes for free  Computational resources are powerful  Examples: NTP is energy expensive, GPS is cost expensive

10. 10 Related Work  Reference Broadcast Synchronization (RBS) [Elson 02]  Conventional receiver-receiver protocol  Reduces nondeterministic delays  Conserves energy via post facto synchronization  Timing synch Protocol for Sensor Networks (TPSN) [Ganeriwal 03]  Conventional sender-receiver protocol  Two operational phases: Level Discovery and Synchronization  Time Diffusion Protocol (TDP) [Su 05]  Achieves a network-wide equilibrium time using an iterative, weighted averaging technique based on diffusion of timing messages

11. 11 Related Work  Analysis of a sender-receiver model [Ghaffar 02]  For known fixed delays, maximum likelihood estimator for clock offset does not exist  Five algorithms: median round delay, minimum round delay, minimum link delay, median phase, average phase.  Minimum link delay algorithm has the lowest variance  Maximum likelihood clock offset estimator for unknown fixed delays [Jeske 05]

12. 12 Clock Model  A computer clock consists of two components  Frequency source  Means of accumulating timing events  Practical clocks are set with limited precision  Frequency sources run at slightly different rates  Frequency of a crystal oscillator varies due to  Initial manufacturing tolerance  Temperature, pressure  Aging

13. 13 Clock Model  A general clock model can be represented by  where is the clock offset, is the clock skew and is the clock drift  Clock synchronization problem  Given the logical clock for a node k in the network, then  is a function of  Target synchronization accuracy  Amount of energy the network is willing to pay

14. 14 1. Node A sends a timing message (Level of Node A and T 1 ) to Node B at T 1. 2. Node B sends an ACK (Level of Node B, T 1, T 2, and T 3 ) to Node A at T 3. With this, Node A calculates the clock offset. Sources of error (time uncertainty) associated with message exchanges o Send time: time spent to construct a message o Access time: delays at MAC layer before actual transmission o Propagation time: time of flight from one node to another o Receive time: time needed for the receiver to receive the message and process it A Sender-Receiver Protocol

15. 15 Observations  Fixed clock offset model is not sufficient in practice  Clock skew correction results in long term synchronization and hence more energy savings  Network delays being asymmetric is a more realistic scenario  Even for the symmetric clock offset only model, better estimation schemes achieving are possible  Minimum Variance Unbiased Estimation (MVUE)  Minimum Mean Square Error Estimation (MMSE)  Lack of analytical performance bounds and metrics  Average RBS error: [Elson 02] or [Ganeriwal 03]?

16. 16 Clock Offset Gaussian Noise Assumption  One motivation comes from experimental basis [Elson 02]  In case of unknown delay distribution, we can evoke Central Limit theorem  Example: for uniform delays, the sum of even two of them starts resembling a Gaussian curve

17. 17 Clock Offset  The likelihood function can be written as  And the clock offset estimate and the CRLB are

18. 18 Clock Offset Exponential Delay Assumption  Random delays often modeled as exponential  Several traces of delay measurements on Internet collected by [Moon 99] fitting an exponential distribution  Conformation of experimental observations with mathematical results  Experimental observations  Minimum link delay algorithm [Paxson 98]  Clock Filter algorithm in NTP [Mills 91]  Mathematical results  Best performance by Minimum link delay algorithm [Ghaffar 02]  ML estimate based on minimum order statistics [Jeske 05]

19. 19 Clock Offset  Likelihood function is given as  ML clock offset estimate is  CRLB is derived as

20. 20 Clock Offset and Skew

21. 21 Clock Offset and Skew Gaussian  Likelihood function with is  Joint ML estimate for clock offset is shown to be where

22. 22 Clock Offset and Skew Gaussian  And for the clock skew  Computationally quite complex  Fixed delay must be known  Open problem: Recursive implementation/update?

23. 23 Clock Offset and Skew Gaussian  Cramer-Rao Lower Bound is expressed as where  Proportional to clock skew squared  Not only dependent on number of synchronization messages but also on the synchronization period

24. 24 Clock Offset and Skew Exponential  The likelihood function in this case is  Four different cases need to be considered Case IKnown Case IIKnownUnknown Case IIIUnknownKnown Case IVUnknown

25. 25 Clock Offset and Skew Exponential Case I: known, known  Constraints  ML estimator

26. 26 Clock Offset and Skew Exponential

27. 27 Clock Offset and Skew Exponential Case II: known, unknown  Constraints  Lemma 1: lies on one of the following curves

28. 28 Clock Offset and Skew Exponential  Lemma 2: lies either on point A or to the left of it (B,C,…)  Lemma 3: To the left of A, boundary of support region is formed by a sequence of curves with decreasing slopes  Lemma 4: is unique and is given by one of

29. 29 Clock Offset and Skew Exponential

30. 30 Clock Offset and Skew Exponential

31. 31 Clock Offset and Skew Exponential Case III: unknown, known  Constraints

32. 32 Clock Offset and Skew Exponential  Lemma 5: Only two points satisfy the constraints  ML estimator has the closed-form expression

33. 33 Clock Offset and Skew Exponential Case IV: unknown, unknown  Constraints  Curves intersect on the line  Over this line, is constrained by

34. 34 Clock Offset and Skew Exponential  Problem can be solved by the application of four lemmas  Final form of the ML estimator is

35. 35 Clock Offset and Skew Exponential

36. 36 Clock Offset and Skew Exponential

37. 37 Simplified Schemes  Fixed delay must be known in Gaussian case  Computational complexity  Further simplification within the same framework is possible suitable for WSNs in case  Synchronization accuracy constraints are not stringent  Energy conservation constraints are strict  One simple scheme is independent of delay distribution involved  Cost paid is slight degradation in estimation quality

38. 38 Utilizing Data Samples 1,N  Better skew estimation for large synchronization period  Utilize only 1 st and last sample differences for eliminating the clock offset  Define  Simplified new model where and are either Gaussian or Laplacian distributed depending on original delay distribution

39. 39 Utilizing Data Samples 1,N Gaussian delays  Likelihood function for highly reduced data set is  ML-Like clock skew estimator is expressed as  CRLB-Like lower bound is  Depends on timestamping “distances”

40. 40 Utilizing Data Samples 1,N Exponential delays  The reduced likelihood function is  ML-Like clock skew estimator can be derived as  CRLB-Like lower bound

41. 41 Utilizing Data Samples 1,N  Simulation results

42. 42 Two Minimum Order Statistics  Motivation  Unknown delay distribution  Small synchronization period  Opening the model equations as  Choose two points as

43. 43 Two Minimum Order Statistics  Joint the two points to obtain the estimate through its slope and intercept  The form of the estimator is  Almost as simple as the clock offset only case  Knowledge of is not required

44. 44 Two Minimum Order Statistics

45. 45 Two Minimum Order Statistics  Simulations results

46. 46 Two Minimum Order Statistics  Computational complexity comparison with the MLE

47. 47 Summary GaussianExponential Offset ModelMLE + CRLBCRLB Offset + Skew ModelMLE + CRLBMLE + Algorithms Offset + Skew ModelUsing First and Last sample ML-Like + LB Using First and Last sample ML-Like + LB Offset + Skew ModelTwo minimum order statistics Algorithm + Computational Complexity

48. 48 Best Linear Unbiased Estimation – Order Statistics  Limited power resources in WSN implies better estimation techniques should be utilized  Results derived so far correspond to symmetric delays, although asymmetry is a more realistic scenario  Best Linear Unbiased Estimation (BLUE) is suboptimal in general due to linearity constraint  What if the linearity constraints are on the order statistics of observed data, instead of the raw observations?

49. 49 Best Linear Unbiased Estimation – Order Statistics  Transforming the data as  Following relations hold for ordered data

50. 50 Best Linear Unbiased Estimation – Order Statistics  The covariance matrix for can be derived as  Its inverse can be found by Gauss-Jordan elimination  Let the ordered observations be represented as

51. 51 Best Linear Unbiased Estimation – Order Statistics Asymmetric Link Delays  The asymmetric linear model can be written as  And the Gauss-Markov theorem implies

52. 52 Best Linear Unbiased Estimation – Order Statistics  The covariance matrix for is  The final expression for is

53. 53 Best Linear Unbiased Estimation – Order Statistics Symmetric Link Delays  The symmetric linear model can be written as  The Gauss-Markov theorem yields the solution

54. 54 Best Linear Unbiased Estimation – Order Statistics  Covariance for is  The expression for is  BLUE-OS for same as the MLE for symmetric link delays

55. 55 Minimum Variance Unbiased Estimation Asymmetric Link Delays  Found by the application of Rao-Blackwell-Lehmann-Scheffe theorem  Likelihood function can be expressed as  According to Neyman-Fisher factorization theorem, the sufficient statistics is

56. 56 Minimum Variance Unbiased Estimation  Notice that  Find such that  Applicable only if is a complete sufficient statistic Only function of is unbiased

57. 57 Minimum Variance Unbiased Estimation  Unbiased estimator of ?  Note that BLUE-OS is unbiased and hence MVUE !  Compensation for asymmetry through  Variance of the clock offset

58. 58 Minimum Variance Unbiased Estimation Symmetric Link Delays  Again applying the Rao-Blackwell-Lehmann-Scheffe theorem, the likelihood function is  More than one unbiased functions of complete statistic?  Through Neyman-Fisher factorization theorem, the actual sufficient statistics is

59. 59 Minimum Variance Unbiased Estimation  is proved to be complete  Unbiased estimator of ?  BLUE-OS is unbiased and hence the MVUE  In symmetric case, the MVUE and BLUE-OS of coincide with MLE  Its variance is

60. 60 Summary Clock Offset MVUE Symmetric DelaysMSE Remarks Asymmetric DelaysMVUE MSE Remarks

61. 61 Explanatory Remarks  Does this discontinuity in clock offset estimates performance make sense?  Which estimator is better when the network delays are slightly symmetric?  MVUE is the best in unbiased class of estimators, not all.  For asymmetric case,

62. 62 Explanatory Remarks  The MLE outperforms the MVUE under the condition  Estimator could be chosen according to the number of synchronization messages if knowledge of is available  Around the point, MLE attains lesser MSE as the link asymmetry decreases, i.e.,

63. 63 Explanatory Remarks  Simulations results

64. 64 Explanatory Remarks  Apparently, adapting between the two estimators a good idea according to since have been obtained too.  Despite the fact that MLE is functionally invariant, considerable amplification of errors occurs due to repeated nonlinear processing  Results are even applicable to Internet time synchronization

65. 65 Explanatory Remarks  As a byproduct, the MVUE of the fixed and mean variable link delays are obtained  Endd-to-end delay measurements are helpful in analyzing network performance  Very useful for applications behaving adaptively based on observed network performance  Continuous media applications, such as audio and video, absorb the delay jitter perceived at receiver for smooth playout of media stream

66. 66 Minimum Mean Square Error Estimation  In general, the MMSE estimator is not realizable due to the dependence of MSE on the unknown parameter  MSE is a sum of variance and bias squared  This dependence usually comes from the bias  Setting the MSE proportional to inverse of the scale parameter cancels the dependent factors

67. 67 Minimum Mean Square Error Estimation  The MMSE estimator comes out to be a function of MVUE  Closed-form expression for  And for mean link delays  MSE of clock offset

68. 68 Minimum Mean Square Error Estimation  The MMSE estimator comes out to be a function of MVUE  Closed-form expression for  And for mean link delays  MSE of clock offset

69. 69 Clock Synchronization for Inactive Nodes  Packet synchronization protocols  Receiver-Receiver (R-R)  Sender-Receiver (S-R)  For WSNs implementing any sender-receiver protocol, the inactive nodes can exploit the timing messages received

70. 70 Clock Synchronization for Inactive Nodes  The model can be represented as

71. 71 Clock Synchronization for Inactive Nodes  Likelihood function, assuming symmetric delays, is  The maximum likelihood estimator is derived as

72. 72 Clock Synchronization for Inactive Nodes

73. 73 Clock Synchronization for Inactive Nodes  The pdf of is obtained as  Cramer-Rao Lower Bound is  Is the ML estimator efficient?

74. 74 Clock Synchronization for Inactive Nodes  An efficient estimator does not exist due to the rule “if an efficient estimator exists, the ML procedure will produce it”.  Simulation results

75. 75 Clock Synchronization for Inactive Nodes  Symmetric delay assumption was less realistic  Better estimation techniques can be employed  Using the transformed data, the linear model can be written as

76. 76 Clock Synchronization for Inactive Nodes and  Hence, the covariance matrix is  Final form of estimator

77. 77 Clock Synchronization for Inactive Nodes  The likelihood function in symmetric case is  MVUE for the clock offset is derived and shown to coincide with BLUE-OS and MLE  Its variance is give by

78. 78 Clock Synchronization for Inactive Nodes  Similarly, for asymmetric link delays, the BLUE-OS is which is also the MVUE  Its variance is given by

79. 79 Clock Synchronization for Inactive Nodes  MMSE estimator can be derived as a function of MVUE  Closed-form expression for  And for mean link delays  MSE of clock offset

80. 80 Clock Offset and Skew in a Receiver-Receiver Protocol  Main sources of errors – send time and channel access time – are removed  A receiver-receiver model can be represented as

81. 81 Clock Offset and Skew in a Receiver-Receiver Protocol  The likelihood function can be expressed as where  Objective function to be maximized is over the constraints

82. 82 Clock Offset and Skew in a Receiver-Receiver Protocol  JML estimator

83. 83 Clock Offset and Skew in a Receiver-Receiver Protocol  Why Gibbs Sampler?  Joint ML estimator is biased  MVUE does not exist since the sufficient statistics depend on unknown parameters  Posterior distribution can be found and clock parameters can be estimated by its mean for better results  Straightforward extension to additional unknown parameters, e.g., clock drift  Posterior distribution involves complex integrations, hence the Markov-Chain Monte Carlo (MCMC) methods

84. 84 Clock Offset and Skew in a Receiver-Receiver Protocol  Algorithm for Gibbs Sampling is to iterate the following with initial values :  After a threshold value, the set behaves as the sample values from the joint posterior

85. 85 Clock Offset and Skew in a Receiver-Receiver Protocol  In the current scenario, the Gibbs Sampler is implemented as follows  Performs better than the JML estimator  Simulated with  MVUE as the lower bound with one parameter known  BLUE as the upper bound due to linearity constraints

86. 86 Clock Offset and Skew in a Receiver-Receiver Protocol  Simulation results

87. 87 Conclusions  General exponential family model  Accumulated error analysis for multihop protocols  Effect of mobility on time synchronization

88. 88 Future Research Directions  General exponential family model  Accumulated error analysis for multihop protocols  Effect of mobility on time synchronization