1. Shadow Configurations: A Network Management Primitive Richard Alimi, Ye Wang, Y. Richard Yang Laboratory of Networked Systems Yale University

2. August 19, 2008Yale LANS / SIGCOMM 20082 Configuration is Complex Source: The Yankee Group, 2004 Source: Juniper Networks, 2008 “... human error is blamed for 50-80% of network outages.” “80% of IT budgets is used to maintain the status quo.”

3. August 19, 2008Yale LANS / SIGCOMM 20083 Configuration is Complex Source: The Yankee Group, 2004 Source: Juniper Networks, 2008 “... human error is blamed for 50-80% of network outages.” “80% of IT budgets is used to maintain the status quo.” Why is configuration hard today?

4. August 19, 2008Yale LANS / SIGCOMM 20084 Simulation & Analysis  Depend on simplified models Network structure Hardware and software  Limited scalability  Hard to access real traffic Configuration Management Today OSPF eBGP VPNs ACLs TE SLAsiBGPTraffic Software Hardware

5. August 19, 2008Yale LANS / SIGCOMM 20085 Configuration Management Today Simulation & Analysis  Depend on simplified models Network structure Hardware and software  Limited scalability  Hard to access real traffic Test networks  Can be prohibitively expensive OSPF eBGP VPNs ACLs TE SLAsiBGPTraffic Software Hardware

6. August 19, 2008Yale LANS / SIGCOMM 20086 Configuration Management Today Simulation & Analysis  Depend on simplified models Network structure Hardware and software  Limited scalability  Hard to access real traffic Test networks  Can be prohibitively expensive OSPF eBGP VPNs ACLs TE SLAsiBGPTraffic Software Hardware Why are these not enough?

7. August 19, 2008Yale LANS / SIGCOMM 20087 Analogy with Programming Programming Program Target System

8. August 19, 2008Yale LANS / SIGCOMM 20088 Analogy with Programming Programming Network Management Program Target System Configs Target Network

9. August 19, 2008Yale LANS / SIGCOMM 20089 Analogy with Databases Databases INSERT... DELETE... UPDATE... INSERT... DELETE... UPDATE... STATE A STATE B

10. August 19, 2008Yale LANS / SIGCOMM 200810 Analogy with Databases Databases Network Management INSERT... DELETE... UPDATE... INSERT... DELETE... UPDATE... STATE A STATE B ip route... ip addr... STATE A ? router bgp... STATE B STATE C router ospf... STATE D

11. August 19, 2008Yale LANS / SIGCOMM 200811 Enter, Shadow Configurations Key ideas  Allow additional (shadow) config on each router  In-network, interactive shadow environment  “Shadow” term from computer graphics OSPF eBGP VPNs ACLs TE SLAsiBGPTraffic Software Hardware

12. August 19, 2008Yale LANS / SIGCOMM 200812 Enter, Shadow Configurations OSPF eBGP VPNs ACLs TE SLAsiBGPTraffic Software Hardware Key Benefits  Realistic (no model)‏  Scalable  Access to real traffic  Transactional Key ideas  Allow additional (shadow) config on each router  In-network, interactive shadow environment  “Shadow” term from computer graphics

13. August 19, 2008Yale LANS / SIGCOMM 200813 Roadmap Motivation and Overview System Basics and Usage System Components  Design and Architecture  Performance Testing  Transaction Support Implementation and Evaluation

14. August 19, 2008Yale LANS / SIGCOMM 200814 What's in the shadow configuration?  Routing parameters  ACLs  Interface parameters  VPNs  QoS parameters Shadow config Real config System Basics

15. August 19, 2008Yale LANS / SIGCOMM 200815 What's in the shadow configuration?  Routing parameters  ACLs  Interface parameters  VPNs  QoS parameters Shadow config Real config Shadow header marked “1” Real header marked “0” System Basics

16. August 19, 2008Yale LANS / SIGCOMM 200816 Example Usage Scenario: Backup Path Verification Primary Backup

17. August 19, 2008Yale LANS / SIGCOMM 200817 Example Usage Scenario: Backup Path Verification Send test packets in shadow

18. August 19, 2008Yale LANS / SIGCOMM 200818 Example Usage Scenario: Backup Path Verification Disable shadow link XX

19. August 19, 2008Yale LANS / SIGCOMM 200819 Example Usage Scenario: Backup Path Verification

20. August 19, 2008Yale LANS / SIGCOMM 200820 Example Usage Scenario: Configuration Evaluation Video Server

21. August 19, 2008Yale LANS / SIGCOMM 200821 Example Usage Scenario: Configuration Evaluation Video Server

22. August 19, 2008Yale LANS / SIGCOMM 200822 Example Usage Scenario: Configuration Evaluation Video Server Duplicate packets to shadow

23. August 19, 2008Yale LANS / SIGCOMM 200823 Roadmap Motivation and Overview System Basics and Usage System Components  Design and Architecture  Performance Testing  Transaction Support Implementation and Evaluation

24. August 19, 2008Yale LANS / SIGCOMM 200824 Design and Architecture Management Control Plane Forwarding Engine Configuration UI BGP OSPF IS-IS Interface 0 Interface 1 Interface 2 Interface 3 FIB

25. August 19, 2008Yale LANS / SIGCOMM 200825 Design and Architecture Management Control Plane Forwarding Engine Configuration UI BGP OSPF IS-IS Shadow-enabled FIB Shadow Bandwidth Control Interface 0 Interface 1 Interface 2 Interface 3

26. August 19, 2008Yale LANS / SIGCOMM 200826 Design and Architecture Management Control Plane Forwarding Engine Configuration UI Shadow Management BGP OSPF IS-IS Shadow-enabled FIB Shadow Bandwidth Control Interface 0 Interface 1 Interface 2 Interface 3

27. August 19, 2008Yale LANS / SIGCOMM 200827 Design and Architecture Management Control Plane Forwarding Engine Configuration UI Shadow Management Commitment BGP OSPF IS-IS Shadow-enabled FIB Shadow Bandwidth Control Interface 0 Interface 1 Interface 2 Interface 3

28. August 19, 2008Yale LANS / SIGCOMM 200828 Design and Architecture Management Control Plane Forwarding Engine Configuration UI Shadow Traffic Control FIB Analysis Debugging Tools Shadow Management Commitment BGP OSPF IS-IS Shadow-enabled FIB Shadow Bandwidth Control Interface 0 Interface 1 Interface 2 Interface 3

29. August 19, 2008Yale LANS / SIGCOMM 200829 Design and Architecture Management Control Plane Forwarding Engine Configuration UI Shadow Traffic Control FIB Analysis Debugging Tools Shadow Management Commitment BGP OSPF IS-IS Shadow-enabled FIB Shadow Bandwidth Control Interface 0 Interface 1 Interface 2 Interface 3

30. August 19, 2008Yale LANS / SIGCOMM 200830 Shadow Bandwidth Control Requirements  Minimal impact on real traffic  Accurate performance measurements of shadow configuration

31. August 19, 2008Yale LANS / SIGCOMM 200831 Shadow Bandwidth Control Requirements  Minimal impact on real traffic  Accurate performance measurements of shadow configuration Supported Modes  Priority  Bandwidth Partitioning  Packet Cancellation

32. August 19, 2008Yale LANS / SIGCOMM 200832 Observation  Content of payload may not important in many network performance testing scenarios  Only payload size may matter Packet Cancellation

33. August 19, 2008Yale LANS / SIGCOMM 200833 Observation  Content of payload may not important in many network performance testing scenarios  Only payload size may matter Idea: only need headers for shadow traffic Piggyback shadow headers on real packets Piggybacked shadow header Packet Cancellation

34. August 19, 2008Yale LANS / SIGCOMM 200834 Packet Cancellation Details Output interface maintains real and shadow queues Packet cancellation scheduling  If real queue non-empty Grab real packet Piggyback shadow header(s) if available  Else if shadow queue non-empty Send full shadow packet

35. August 19, 2008Yale LANS / SIGCOMM 200835 Commitment Objectives  Smoothly swap real and shadow across network Eliminate effects of transient states due to config changes  Easy to swap back

36. August 19, 2008Yale LANS / SIGCOMM 200836 Commitment Objectives  Smoothly swap real and shadow across network Eliminate effects of transient states due to config changes  Easy to swap back Issue  Packet marked with shadow bit 0 = Real, 1 = Shadow  Shadow bit determines which FIB to use  Routers swap FIBs asynchronously  Inconsistent FIBs applied on the path

37. August 19, 2008Yale LANS / SIGCOMM 200837 Commitment Protocol Idea: Use tags to achieve consistency  Temporary identifiers Basic algorithm has 4 phases

38. August 19, 2008Yale LANS / SIGCOMM 200838 Commitment Protocol Idea: Use tags to achieve consistency  Temporary identifiers Basic algorithm has 4 phases  Distribute tags for each config C-old for current real config C-new for current shadow config 0 0 0 0 1 1 0: C-old 1: C-new 1 0 1 0 1 0 0

39. August 19, 2008Yale LANS / SIGCOMM 200839 Commitment Protocol Idea: Use tags to achieve consistency  Temporary identifiers Basic algorithm has 4 phases  Distribute tags for each config C-old for current real config C-new for current shadow config  Routers mark packets with tags C-old C-new C-old C-new C-old C-new 1 0 1 0 1 0 0

40. August 19, 2008Yale LANS / SIGCOMM 200840 C-old C-new C-old C-new C-old C-new 0: C-new 1: C-old 1 0 1 0 1 0 1 Commitment Protocol Idea: Use tags to achieve consistency  Temporary identifiers Basic algorithm has 4 phases  Distribute tags for each config C-old for current real config C-new for current shadow config  Routers mark packets with tags  Swap configs (tags still valid)‏

41. August 19, 2008Yale LANS / SIGCOMM 200841 Commitment Protocol Idea: Use tags to achieve consistency  Temporary identifiers Basic algorithm has 4 phases  Distribute tags for each config C-old for current real config C-new for current shadow config  Routers mark packets with tags  Swap configs (tags still valid)‏  Remove tags from packets Resume use of shadow bit 0 0 1 0 1 0 1 0 1

42. August 19, 2008Yale LANS / SIGCOMM 200842 Commitment Protocol Idea: Use tags to achieve consistency  Temporary identifiers Basic algorithm has 4 phases  Distribute tags for each config C-old for current real config C-new for current shadow config  Routers mark packets with tags  Swap configs (tags still valid)‏  Remove tags from packets Resume use of shadow bit  For more details, see paper 0 0 1 0 1 0 1 0 1

43. August 19, 2008Yale LANS / SIGCOMM 200843 Roadmap Motivation and Overview System Basics and Usage System Components  Design and Architecture  Performance Testing  Transaction Support Implementation and Evaluation

44. August 19, 2008Yale LANS / SIGCOMM 200844 Implementation Kernel-level (based on Linux 2.6.22.9)‏  TCP/IP stack support  FIB management  Commitment hooks  Packet cancellation Tools  Transparent software router support (Quagga + XORP)‏  Full commitment protocol  Configuration UI (command-line based)‏ Evaluated on Emulab (3Ghz HT CPUs)‏

45. August 19, 2008Yale LANS / SIGCOMM 200845 Static FIB  300B pkts  No route caching With FIB updates  300B pkts @ 100Mbps  1-100 updates/sec  No route caching Static FIB 300B pkts No route caching

46. August 19, 2008Yale LANS / SIGCOMM 200846 FIB storage overhead for US Tier-1 ISP Evaluation: Memory Overhead

47. August 19, 2008Yale LANS / SIGCOMM 200847 Evaluation: Packet Cancellation Accurate streaming throughput measurement  Abilene topology  Real transit traffic duplicated to shadow  Video streaming traffic in shadow

48. August 19, 2008Yale LANS / SIGCOMM 200848 Evaluation: Packet Cancellation Limited interaction of real and shadow  Intersecting real and shadow flows CAIDA traces  Vary flow utilizations

49. August 19, 2008Yale LANS / SIGCOMM 200849 Evaluation: Packet Cancellation Limited interaction of real and shadow  Intersecting real and shadow flows CAIDA traces  Vary flow utilizations

50. August 19, 2008Yale LANS / SIGCOMM 200850 Evaluation: Commitment Applying OSPF link-weight changes  Abilene topology with 3 external peers Configs translated to Quagga syntax Abilene BGP dumps

51. August 19, 2008Yale LANS / SIGCOMM 200851 Evaluation: Commitment Applying OSPF link-weight changes  Abilene topology with 3 external peers Configs translated to Quagga syntax Abilene BGP dumps Reconvergence in shadow

52. August 19, 2008Yale LANS / SIGCOMM 200852 Conclusion and Future Work Shadow configurations is new management primitive  Realistic in-network evaluation  Network-wide transactional support for configuration Future work  Evaluate on carrier-grade installations  Automated proactive testing  Automated reactive debugging

53. August 19, 2008Yale LANS / SIGCOMM 200853 Thank you!

54. August 19, 2008Yale LANS / SIGCOMM 200854 Backup Slides

55. August 19, 2008Yale LANS / SIGCOMM 200855 Evaluation: Router Maintenance Setup  Abilene topology with 3 external peers Configs translated to Quagga syntax Abilene BGP dumps