1. Fine-Grained Network Time Synchronization Using Reference Broadcasts Jeremy Elson, Lew Girod, and Deborah Estrin University of California, Los Angeles OSDI 2002 - Boston, MA Used with permission of author

2. The bigger picture Isn’t this a solved problem by now??? –NTP, many other clock agreement algorithms, MACs with sync built in (802.11), time broadcasts (GPS, WWVB), high-stability oscillators (Rubidium, Cesium) If this isn’t the Internet: –Important assumptions no longer hold (fewer resources -- such as energy, good connectivity, infrastructure, size, and cost -- are available …) –Sensor apps have stronger requirements (…but we have to do better than the Internet anyway) BUT...

3. A palette of sync methods Time Sync Parameter Space: (max error, lifetime, scope, etc.) Application Requirement Available Sync Methods Better Goal: make the set rich enough to minimize waste

4. A palette of sync methods Time Sync Parameter Space: (max error, lifetime, scope, etc.) Application Requirement Better Goal: make the set rich enough to minimize waste Ideally, methods should be tunable

5. Time Definitions Clock stability – how well it maintains a constant frequency –Short term – temperature –Long term – aging of oscillator Clock accuracy – how well its frequency and time compare with standard Offset – time difference between 2 clocks Skew – frequency difference between 2 clocks

6. Traditional sync SenderReceiver At the tone: t=1 NIC Physical Media NIC Send time Access Time Propagation Time Receive Time Problem: Many sources of unknown, nondeterministic latency between timestamp and its reception

7. Reference Broadcasts SenderReceiver NIC Physical Media NIC Propagation Time Receive Time Sync 2 receivers with each other, NOT sender with receiver Receiver NIC I saw it at t=4 I saw it at t=5

8. NIC Sender Receiver 1 Receiver 2 Critical Path NIC Sender Receiver Critical Path Time RBS reduces error by removing much of it from the critical path Traditional critical path: From the time the sender reads its clock, to when the receiver reads its clock RBS: Only sensitive to the differences in receive time and propagation delay

9. Receiver Determinism 1st testbed: Berkeley motes with narrowband (19.2K) radios

10. Test to determine if observed data set satisfies specified distribution –In this case, Gaussian with = 0, = 11.1 Build histogram of k cells and compare observed against expected frequencies in each cell D =  (o i –e i ) 2 /e i Chi-squared Test   k 1 Exact fit D = 0 D < X 2 [1-a,k-1] table A.5 can not reject null hypothesis

11. Why Is Gaussian So Popular? If x i ~ N(  ,   ) and all x i independent, then  i x i is normal with mean  i  i and variance    i 2  i 2 Sum of large no. of independent observations from any distribution is itself normal (Central Limit Theorem)  Experimental errors can be modeled as normal distribution.

12. Well-Behaved = Good Well behaved distributions are useful –Error can be reduced statistically, by sending multiple pulses over time and averaging –Also, easier to model/simulate Still ignoring clock skew –Server broadcasts m reference beacons –Each of n receivers records local time of each m refs –Exchange: Offset[i,j] = 1/m  T j,k – T i,k ) k=1 m

13. Problem: Clock Skew It takes time to send multiple pulses By the time we do, clocks will have drifted So: don’t average; fit a line instead

14. Time

15. Linear Regression Models What is a (good) model? Estimating model parameters Allocating variation Confidence intervals for regressions Verifying assumptions visually © 1998, Geoff Kuenning

16. What Is a (Good) Model? For correlated data, model predicts response given an input Model should be equation that fits data Standard definition of “fits” is least-squares –Minimize squared error –While keeping mean error zero –Minimizes variance of errors © 1998, Geoff Kuenning

17. Least-Squared Error If then error in estimate for x i is Minimize Sum of Squared Errors (SSE) Subject to the constraint. e i = y i - y i. © 1998, Geoff Kuenning

18. Estimating Model Parameters Best regression parameters are where Note error in book! © 1998, Geoff Kuenning

19. Parameter Estimation Example Execution time of a script for various loop counts: = 6.8, = 2.32,  xy = 88.54,  x 2 = 264 b 0 = 2.32  (0.29)(6.8) = 0.35 © 1998, Geoff Kuenning

20. Graph of Parameter Estimation Example © 1998, Geoff Kuenning

21. Comparison to NTP Second implementation: –Compaq IPAQs (small Linux machines) –11mbit 802.11 PCMCIA cards Ran NTP, RBS-Userspace, RBS-Kernel –NTP synced to GPS clock every 16 secs –NTP with phase correction, too; it did worse (!) In each case, asked 2 IPAQs to raise a GPIO line high at the same time; differences measured with logic analyzer

22. How NTP works Multiple synchronization peers provide redundancy and diversity Clock filters select best from a window of eight clock offset samples Intersection and clustering algorithms pick best subset of servers believed to be accurate and fault-free Combining algorithm computes weighted average of offsets for best accuracy Phase/frequency-lock feedback loop disciplines local clock time and frequency to maximize accuracy and stability NTP Messages Peer 1 Peer 2 Filter 1 Peer 3 Filter 2 Filter 3 Intersection and Clustering Algorithms Combining Algorithm Loop Filter VFO Timestamps P/F-Lock Loop © Mills

23. Clock Resolution

24. RBS degraded slightly (6us to 8us); NTP degraded severely (51us to 1542us)

25. “Here 0 sec after blue pulse!” “Here 1 sec after blue pulse!” “Here 3 sec after red pulse!” “Here 1 sec after red pulse!” Multi-Hop RBS Some nodes broadcast RF synchronization pulses Receivers in a neighborhood are synced by using the pulse as a time reference. (The pulse senders are not synced.) Nodes that hear both can relate the time bases to each other “Red pulse 2 sec after blue pulse!”

26. 1 3 2 A 4 8 C 5 7 6 B 10 D 11 9 Time Routing Consider a physical topology consisting of broadcasters (A, B, C..) and receivers (1, 2, 3…) (In reality, a node can play both roles)

27. 1 3 2 A 4 8 C 5 7 6 B 10 D 11 9 1 3 2 4 8 5 7 6 1011 9 Time Routing The physical topology can be easily converted to a logical topology; links represent possible clock conversions Use shortest path search to find a “time route”; Edges can be weighted by error estimates

28. Multi-Hop RBS 1.85 +/- 1.28 2.73 +/- 1.91 2.73 +/- 2.42 3.68 +/- 2.57 Error (and std dev) over multiple hops, in usec

29. Summary RBS can improve accuracy by removing sender from the critical path Multi-hop algorithm can extend RBS property across broadcast domains, and to external standards such as UTC Implemented on 4 different CPU/radio platforms; no MAC tinkering required Facilitates post-facto sync (save energy by only syncing after an event of interest) and peer to peer sync (no global timescale)

30. Applications Acoustic Ranging Collaborative Signal Detection etc... Future work: –Use higher precision clocks (e.g. Pentium TSC) –Better outlier rejection, weighting

31. Thank you!