Fine-Grained Network Time Synchronization Using Reference Broadcasts Jeremy Elson, Lew Girod, and Deborah Estrin University of California, Los Angeles OSDI Boston, MA Used with permission of author

The bigger picture Isn’t this a solved problem by now??? –NTP, many other clock agreement algorithms, MACs with sync built in (802.11), time broadcasts (GPS, WWVB), high-stability oscillators (Rubidium, Cesium) If this isn’t the Internet: –Important assumptions no longer hold (fewer resources -- such as energy, good connectivity, infrastructure, size, and cost -- are available …) –Sensor apps have stronger requirements (…but we have to do better than the Internet anyway) BUT...

A palette of sync methods Time Sync Parameter Space: (max error, lifetime, scope, etc.) Application Requirement Available Sync Methods Better Goal: make the set rich enough to minimize waste

A palette of sync methods Time Sync Parameter Space: (max error, lifetime, scope, etc.) Application Requirement Better Goal: make the set rich enough to minimize waste Ideally, methods should be tunable

Time Definitions Clock stability – how well it maintains a constant frequency –Short term – temperature –Long term – aging of oscillator Clock accuracy – how well its frequency and time compare with standard Offset – time difference between 2 clocks Skew – frequency difference between 2 clocks

Traditional sync SenderReceiver At the tone: t=1 NIC Physical Media NIC Send time Access Time Propagation Time Receive Time Problem: Many sources of unknown, nondeterministic latency between timestamp and its reception

Reference Broadcasts SenderReceiver NIC Physical Media NIC Propagation Time Receive Time Sync 2 receivers with each other, NOT sender with receiver Receiver NIC I saw it at t=4 I saw it at t=5

NIC Sender Receiver 1 Receiver 2 Critical Path NIC Sender Receiver Critical Path Time RBS reduces error by removing much of it from the critical path Traditional critical path: From the time the sender reads its clock, to when the receiver reads its clock RBS: Only sensitive to the differences in receive time and propagation delay

Receiver Determinism 1st testbed: Berkeley motes with narrowband (19.2K) radios

Test to determine if observed data set satisfies specified distribution –In this case, Gaussian with = 0, = 11.1 Build histogram of k cells and compare observed against expected frequencies in each cell D =  (o i –e i ) 2 /e i Chi-squared Test   k 1 Exact fit D = 0 D < X 2 [1-a,k-1] table A.5 can not reject null hypothesis

Why Is Gaussian So Popular? If x i ~ N(  ,   ) and all x i independent, then  i x i is normal with mean  i  i and variance    i 2  i 2 Sum of large no. of independent observations from any distribution is itself normal (Central Limit Theorem)  Experimental errors can be modeled as normal distribution.

Well-Behaved = Good Well behaved distributions are useful –Error can be reduced statistically, by sending multiple pulses over time and averaging –Also, easier to model/simulate Still ignoring clock skew –Server broadcasts m reference beacons –Each of n receivers records local time of each m refs –Exchange: Offset[i,j] = 1/m  T j,k – T i,k ) k=1 m

Problem: Clock Skew It takes time to send multiple pulses By the time we do, clocks will have drifted So: don’t average; fit a line instead

Time

Linear Regression Models What is a (good) model? Estimating model parameters Allocating variation Confidence intervals for regressions Verifying assumptions visually © 1998, Geoff Kuenning

What Is a (Good) Model? For correlated data, model predicts response given an input Model should be equation that fits data Standard definition of “fits” is least-squares –Minimize squared error –While keeping mean error zero –Minimizes variance of errors © 1998, Geoff Kuenning

Least-Squared Error If then error in estimate for x i is Minimize Sum of Squared Errors (SSE) Subject to the constraint. e i = y i - y i. © 1998, Geoff Kuenning

Estimating Model Parameters Best regression parameters are where Note error in book! © 1998, Geoff Kuenning

Parameter Estimation Example Execution time of a script for various loop counts: = 6.8, = 2.32,  xy = 88.54,  x 2 = 264 b 0 = 2.32  (0.29)(6.8) = 0.35 © 1998, Geoff Kuenning

Graph of Parameter Estimation Example © 1998, Geoff Kuenning

Comparison to NTP Second implementation: –Compaq IPAQs (small Linux machines) –11mbit PCMCIA cards Ran NTP, RBS-Userspace, RBS-Kernel –NTP synced to GPS clock every 16 secs –NTP with phase correction, too; it did worse (!) In each case, asked 2 IPAQs to raise a GPIO line high at the same time; differences measured with logic analyzer

How NTP works Multiple synchronization peers provide redundancy and diversity Clock filters select best from a window of eight clock offset samples Intersection and clustering algorithms pick best subset of servers believed to be accurate and fault-free Combining algorithm computes weighted average of offsets for best accuracy Phase/frequency-lock feedback loop disciplines local clock time and frequency to maximize accuracy and stability NTP Messages Peer 1 Peer 2 Filter 1 Peer 3 Filter 2 Filter 3 Intersection and Clustering Algorithms Combining Algorithm Loop Filter VFO Timestamps P/F-Lock Loop © Mills

Clock Resolution

RBS degraded slightly (6us to 8us); NTP degraded severely (51us to 1542us)

“Here 0 sec after blue pulse!” “Here 1 sec after blue pulse!” “Here 3 sec after red pulse!” “Here 1 sec after red pulse!” Multi-Hop RBS Some nodes broadcast RF synchronization pulses Receivers in a neighborhood are synced by using the pulse as a time reference. (The pulse senders are not synced.) Nodes that hear both can relate the time bases to each other “Red pulse 2 sec after blue pulse!”

1 3 2 A 4 8 C B 10 D 11 9 Time Routing Consider a physical topology consisting of broadcasters (A, B, C..) and receivers (1, 2, 3…) (In reality, a node can play both roles)

1 3 2 A 4 8 C B 10 D Time Routing The physical topology can be easily converted to a logical topology; links represent possible clock conversions Use shortest path search to find a “time route”; Edges can be weighted by error estimates

Multi-Hop RBS / / / / Error (and std dev) over multiple hops, in usec

Summary RBS can improve accuracy by removing sender from the critical path Multi-hop algorithm can extend RBS property across broadcast domains, and to external standards such as UTC Implemented on 4 different CPU/radio platforms; no MAC tinkering required Facilitates post-facto sync (save energy by only syncing after an event of interest) and peer to peer sync (no global timescale)

Applications Acoustic Ranging Collaborative Signal Detection etc... Future work: –Use higher precision clocks (e.g. Pentium TSC) –Better outlier rejection, weighting

Thank you!