1. Computer Engineering and Networks Laboratory How Was Your Journey? Uncovering Routing Dynamics in Deployed Sensor Networks with Multi-hop Network Tomography Matthias Keller, Jan Beutel, Lothar Thiele SenSys 2012, Toronto, ON, Canada

2. Swiss Federal Institute of Technology 2 Multi-Hop Network Tomography  Multi-hop data collection  Tree-based routing protocol  Low-power operation

3. Swiss Federal Institute of Technology 3 Multi-Hop Network Tomography B A C S B: Node C suffers from a weak link A: Node C was disconnected D: Weak link at upstream node C: Backpressure End-to-end delay Packets from Why did the end-to-end packet delay increase? C

4. Swiss Federal Institute of Technology 4 Multi-Hop Network Tomography Problem: Missing Global Network State Global network state S B C D E F A S B C D E F A Detailed analysis requires network state from all nodes at all times Tomography Only partial information from packet sources

5. Swiss Federal Institute of Technology 5 Multi-Hop Network Tomography Exclusion of reordering Packet correlation S Incoming packets  Information from packet source  Order of arrival at the sink  Per-packet network path  Per-hop arrival order  Per-hop arrival times S B C D E F A 1 1 1 2 2

6. Swiss Federal Institute of Technology 6 Multi-Hop Network Tomography System Model S B C A 1 1 1 22 Sensor nodes  Single FIFO queue  Generate packets  Forward packets of other nodes Sequence number Source address First hop receiver Sojourn time + Payload + Set once at sourceUpdated in network

7. Swiss Federal Institute of Technology 7 Multi-Hop Network Tomography Information Reconstruction S C 1 1 1 2 2 1 2 3 4 Start at packet source Go to next hop At this hop: Locate two anchor packets 1.Last packet generated before our arrival 2.First packet generated after our arrival Extract next hop from anchor packets A B Base for anchor packet selection For every packet:

8. Swiss Federal Institute of Technology 8 Multi-Hop Network Tomography Packet Correlation Example S B C A 1 1 1 2 2 B Next hop S 1 A Anchor packets 2 1 1 2 Timing + next hop Timing + next hop For packet : 1 Select anchor packets  Per-packet network path  Per-hop arrival order  Per-hop arrival times

9. Swiss Federal Institute of Technology 9 Multi-Hop Network Tomography The Problem of Path Changes S B C A D E 1 12 1 1212 12 1 1 Path change! 121 111 2 111 2 121 1 Three orderings are possible at the sink: correct ordering incorrect ordering

10. Swiss Federal Institute of Technology 10 Multi-Hop Network Tomography Exclusion of Reordering “Reliable” packets Packet stream Not “reliable” packets Packet classification  Guaranteed to not have been reordered  Ready for packet correlation  May have been reordered  Removed from tomography

11. Swiss Federal Institute of Technology 11 Multi-Hop Network Tomography Definition: Reliable Packet A packet k is reliable, if it fulfills two properties: From our observations at the sink, we can guarantee that (i)packet k can only have travelled along exactly one path, and that (ii)the order relation between packet k and any other reliable packet is consistent along all packet queues in the network including the sink.

12. Swiss Federal Institute of Technology 12 Multi-Hop Network Tomography Determining a Reliable Set 1 2 3 4 For every packet: Start at packet source Go to next hop Determine possible anchor packets (worst-case analysis)  Single next hop receiver?  Arrived at the sink in order? Extract next hop receiver Stop on error “Reliable” if next hop is sink

13. Swiss Federal Institute of Technology 13 Multi-Hop Network Tomography Verification of Single Possible Path Idea: At each hop, verify that packet can only have left to one single next hop Problem: Per-hop timing is yet unknown Per-hop worst-case analysis: EEE? Window of possible arrival EEAA EEEE Single possible next receiver Multiple possible next receivers Uncertainty due to lost information Next hop Next hop Next hop 1) 2) 3) B A E

14. Swiss Federal Institute of Technology 14 Multi-Hop Network Tomography Verification of Consistent Ordering Generally hard problem, easier after single path verification 3 12 EEE 12 3 1 1 Packets that can only have left to a single next hop are always surrounded by at least two potential anchor packets that point to the same next hop 1 2 Next hop Anchor packets Forwarded traffic Theorem (proof in paper): If all potential anchor packets arrived at the sink in order, also forwarded packets were not reordered at this hop Guaranteed to not have been reordered at E 1 1

15. Swiss Federal Institute of Technology 15 Multi-Hop Network Tomography Validation & Evaluation Testbed experiments for comparison with ground truth  90 TMote Sky nodes running CTP Noe/LPL (TWIST)  25 TinyNode nodes running Dozer (FlockLab) Real-world deployments (PermaSense)  >140 million packets Simulation in Castalia (100-hop line)  Explore scaling properties and limitations a b c # of packets with reconstructed path information # of packets received Reconstructed packets =

16. Swiss Federal Institute of Technology 16 Multi-Hop Network Tomography Testbed Tomography Results Reconstructed information is correct in all cases CTP Noe 90 Tmote Sky nodes Dozer 25 TinyNode nodes Inter-packet interval (IPI)

17. Swiss Federal Institute of Technology 17 Multi-Hop Network Tomography PermaSense Deployments 10-40 TinyNode nodes running Dozer Data yield >98%  Matterhorn, 2008, >78 million received packets  Jungfraujoch, 2009, >48 million received packets  Dirruhorn, 2010, >20 million received packets  Aiguille du Midi, 2012

18. Swiss Federal Institute of Technology 18 Multi-Hop Network Tomography Deployment Tomography Results Matterhorn: 99.5% Artifacts of unintentionally removed and now missing packets in data repository Jungfraujoch: 91.5% Dirruhorn: 93.7%

19. Swiss Federal Institute of Technology 19 Multi-Hop Network Tomography Sensitivity of Performance to Data Yield CTP Noe (testbed) Dozer (testbed) CTP, 100 hop line (simulation) Dozer, large delays (testbed)

20. Swiss Federal Institute of Technology 20 Multi-Hop Network Tomography Accuracy of Per-Hop Arrival Times IPI15 sec30 sec120 sec CTP Noe<1 sec 1 sec Dozer15 sec30 sec84 sec 90 th Percentile of Arrival Time Uncertainty Accuracy of obtained arrival times is not a function of the packet delay, but upper bound by the IPI [Distance between earliest and latest arrival at a node]

21. Swiss Federal Institute of Technology 21 Multi-Hop Network Tomography Stepping Stone for Passive Monitoring Non-intrusive information reconstruction at the sink is possible Testbed experiments with CTP and Dozer proof the correctness of reconstructed data Results from very large data sets confirm applicability of the approach to real-world problems Accuracy of tomography is sensitive to the amount of data (yield), but not to its age (packet delay) www.permasense.ch