1. Control Plane Issues in the Internet: Personal Perspective 2005.4.11. Monday Microsoft Research Asia Beijing, China Sue B. Moon Division of Computer Science Dept. of EECS KAIST

2. 2 Overview Personal Perspective –Single-Hop Delay –Point-to-Point Delay –Routing Anomaly –Path Multiplicity as a Value-Added Service

3. 3 Personal Experience at Sprint When I first arrived, I heard … –“No loss” on Sprint backbone network –“Almost no delay” –“Cadillac brand of IP service”

4. 4 Monitors in San Jose PoP * All monitored links are OC3

5. 5 Min/Avg/Max Delay per Minute

6. 6 Link Utilization

7. 7 Single-Hop Delay Distribution

8. 8 Delay w/o Transmission Time (TT)

9. 9 Minimum Router Transit Time (MRTT)

10. 10 Is the queue work-conserving?

11. 11 Delay w/o TX and MRTT

12. 12 Min/Avg/Max Delay without Cisco Router Idiosyncracies

13. 13 Summary of Single-Hop Delay Packet size is a major factor Non-work-conserving behavior of a router is a main cause behind large delay (> 1ms) Not much queueing observed

14. 14 Point-to-Point Delay

15. 15 Delay Distributions Data Set 3

16. 16 Hourly Delay Distributions Data Set 3

17. 17 Data Set 3 Identification of Constant Factors: Multi-Paths Equal Cost Multi Paths (ECMP) –Src/Dst addresses, Router ID Min delay of src/dst flow (Data Set 3) Path 1 Path 2 Path 3

18. 18 Three Paths Connectivity Data Set 3 28ms 32ms 34ms Fiber prop.delay

19. 19 Path 1 Path Separation of Data Set 3 TTL difference Minimum delay of flow (src ip, dst ip)

20. 20 Identification of Constant Factors: Packet Size Path transit time –Propagation + packet processing (packet size)  

21. 21 Removing Constant Factors Data Set 3 Path1

22. 22 Variable Delay: Bulk Data Set 3, Path 1

23. 23 Variable Delay: Bulk (cont ’ d) Data Set 3

24. 24 Impact of Bottleneck Link Load 90

25. 25 Variable Delay Revisited: Tail Data Set 3, Path 1

26. 26 Peaks in Variable Delay

27. 27 Closer Look Queue Build up & Drain

28. 28 Summary of Pt-to-Pt Delay Not much queueing most of the time Severe congestion when bottleneck link utililization > 90% Congestion periods longer than 1 sec –Exact causes unknown –Possible causes Route changes

29. 29 Routing Loop

30. 30 Issues in "Good" Routing Misbehaving routing protocols –BGP misconfigurations –Pathological behaviors –Frequent changes Even under normal circumstances –Transient behaviors –Inter/intra-domain routing not well understood

31. 31 Scenario for a Transient Routing Loop In Normal Operation

32. 32 When a link fails, R1 is the first to detect.

33. 33 R3 is updated before R2.

34. 34 Finally R2 is updated, and the loop is resolved.

35. 35 CDF of Routing Loop Duration in Time

36. 36 VoIP experimental setup [Boutremans2002] Traffic injected in the network: –200 byte UDP packets –every 5ms. Packets captured and timestamped at end-systems. Traceroute runs continuously during the experiment. Induced link failures on purpose to evalute convergence time and impact on e2e connections

37. 37 Information Sources IS-IS & BGP listener logs Router logs from both ends of “ failing ” links Controlled bi-directional VoIP traffic between Reston and ATL SNMP data

38. 38 Delays (1 sec timescale) 2 links down ~ 3.4ms 2 links up ~ 2.6ms 3 links down 3 links up

39. 39 When the two interfaces went down … 6.6 seconds

40. 40 When three links came back up For 30 secs packets follow a shorter path Traffic “black-holed” for 1.745 seconds Traffic “black-holed” for 0.975 seconds

41. 41 Approaches To Fix It Fine-tuning parameters –Timer values [Alattinoglu2002] Modify Routing Protocols –Suppress advertisement and perform local rerouting using a backwarding table [Lee04] –Centralized path computation [Feamster04,Rexford04]

42. 42 Our Approach Key Idea: –Find disjoint overlay path and send duplicate packets Assumptions –Sender and receiver both within an AS –Bidirectional link weights –Extra income for extra b/w consumption Pros and cons –Advantages No modification to current infrastructure Selective use by only those that need it –Disadvantages Extra b/w consumption

43. 43 Provisioning for Interactive Streaming Interactive Streaming –Not a driving force behind b/w –A candidate for growing revenue Examples –VoIP gradually taking over PSTN traffic –Remote video viewing at door by cell phone –Online game traffic "Good" routing more important than bandwidth

44. 44 Basic Ideas source destination candidate relay nodes!!!

45. 45 Resilient to Failures

46. 46 What I have learned … No loss, almost no delay –Almost. I gained insight into causes behind Debunking the myths [Odlyzko2005] –Streaming real-time traffic –QoS –Content is king –Usage-sensitive pricing

47. 47 Other Issues Tackled Traffic Matrix Estimation –Inspired by tomography in other fields –Before arrival of efficient NetFlow Network Anomaly Detection –NIDS, IDS => PCA-based global monitoring Optimization –Cross-layer resource allocation

48. 48 Future Work Personal perspective –More into creating value-added services –MPLS/VPN performance issues

49. 49 Acknowledgements Thank D. Papagiannaki, B.-Y. Choi, U. Hengartner, C. Boutresmans, G. Iannaccone, and M. Cha for help with the slides.