1. Natale Ruello Technical Marketing Engineer – Nexus 7000 OTV Technology Introduction

2. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 2 Questions to ask on DC Interconnect  How many DC locations (today/future)?  What type of connectivity between the sites for Ethernet/IP (fiber, lambda, L2 service, IP, mix)?  What type of connectivity between sites for Storage (fiber, lambda, FCIP, NAS)  What are the distances between the sites?  What’s the bandwidth requirements between sites?  What are the drivers for L2 extension? (clusters, heartbeats, VM mobility, legacy applications)  What’s the number of VLANs to be extended?  What’s the total number of MACs in the VLANs that need extension?  Where is traffic coming in/going out of the DCs?  Are there stateful services like firewalling/load balancing deployed? Important questions to ask your customer

3. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 3 Cost Optimization Adaptability Availability Addressing Business Goals with LAN Extensions Service Velocity and On-demand Capacity 99.999% Global Availability Maximize Asset Utilization Streamline Operations & Reduce OPEX Enable Distributed Clusters to improve Application Availability without compromising Network Resiliency Unleash Compute Virtualization beyond a single physical data center for fast service and capacity additions Supports migration of workloads and consolidation of servers across locations to avoid power/cooling hot spots or compute/network idleness Enables improved change management methods across multiple physical locations Non-disruptive model for minimal operational overhead Business GoalsLAN ExtensionsAttributes

4. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 4 Enabling IT Solutions with LAN extensions Cluster Vmotion MSCS Cluster Solaris Sun Cluster Enterprise RAC (Real Appl.Cluster) HACMP Legato Automated Availability Mgr Metro Cluster Metrocluster BACnet (building automation/control)

5. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 5 Overlay Transport Virtualization (OTV) O V Overlay - A solution that is independent of the infrastructure technology and services, flexible over various inter-connect facilities Transport - Transporting services for layer 2 and layer 3 Ethernet and IP traffic Virtualization - Provides virtual connections, connections that are in turn virtualized and partitioned into VPNs, VRFs, VLANs OTV LAN Extensions OTV delivers a virtual L2 transport T

6. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 6 Challenges with LAN Extensions Real Problems Solved by OTV  Extensions over any transport (IP, MPLS)  Failure Boundary Preservation  Site independence / Isolation  Optimal BW utilization (no head-end replication)  Resiliency/multi-homing  Built-in end-to-end Loop Prevention  Multi-site connectivity (inter and intra DC)  Scalability  VLANs, Sites, MACs  ARP, Broadcasts/Floods  Operations Simplicity  Topology Flexibility South Data Center North Data Center North Data Center Fault Domain LAN Extension Only 5 CLI commands

7. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 7 Traditional Layer 2 VPNs EoMPLS VPLS Dark Fiber

8. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 8 Flooding Behavior x2x2 Site A Site B Site C MAC 1 propagation MAC 1  Traditional Layer 2 VPN technologies rely on flooding to propagate MAC reachability.  The flooding behavior causes failures to propagate to every site in the L2-VPN.  A solution that provides layer 2 connectivity, yet restricts the reach of the flood domain is necessary in order to contain failures and preserve the resiliency.

9. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 9 Pseudo-wires Maintenance  Before any learning can happen a full mesh of pseudo-wires/tunnels must be in place.  For N sites, there will be N*(N-1)/2 pseudo-wires. Complex to add/remove sites.  Head-end replication for multicast and broadcast. Sub-optimal BW utilization.  A simple overlay protocol with built-in functionality and point-to-cloud provisioning is key to reducing the cost of providing this connectivity

10. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 10 Multi-Homing L2 Site L2 VPN Active  Require additional protocols to support Multi-homing.  STP is often extended across the sites of the Layer 2 VPN. Very difficult to manage as the number of sites grows.  Malfunctions on one site will likely impact all sites on the VPN.  A solution that natively provides automatic detection of multi-homing without the need of extending the STP domains is key.

11. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 11 What can be improved  Data Plane Learning  Control Plane Learning Moving to a Control Plane protocol that proactively advertises MAC addresses and their reachability instead of the current flooding mechanism.  Pseudo-wires and Tunnels  Dynamic Encapsulation No static tunnel or pseudo-wire configuration required. Optimal replication of traffic done closer to the destination, which translates into much more efficient bandwidth utilization in the core.  Multi-Homing  Native Built-in Multi-homing Ideally a multi-homed solution should allow load balancing of flows within a single VLAN across the active devices in the same site, while preserving the independence of the sites. STP confined within the site (each site with its own STP Root bridge)

12. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 12 Overlay Transport Virtualization Technology Pillars Protocol Learning Built-in Loop Prevention Preserve Failure Boundary Seamless Site Addition/Removal Automated Multi-homing Dynamic Encapsulation No Pseudo-Wire State Maintenance Optimal Multicast Replication OTV is a “MAC in IP” technique for supporting Layer 2 VPNs OVER ANY TRANSPORT. Multi-point Connectivity Point-to-Cloud Model

13. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 13 OTV at a Glance  Ethernet traffic between sites is encapsulated in IP: “MAC in IP”  Dynamic encapsulation based on MAC routing table  No Pseudo-Wire or Tunnel state maintained West Site West Site East Site East Site OTV MACIF MAC1Eth1 MAC2IP B MAC3IP B IP A IP B EncapDecap Ethernet Frame IP packet Ethernet Frame MACIF MAC1IP A MAC2Eth 1 MAC3Eth 2 Communication between MAC1 (West) and MAC2 (East)

14. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 14 Eth 4 Eth 3 MAC TABLE VLANMACIF 100MAC 1Eth 2 100MAC 2Eth 1 100MAC 3IP B 100MAC 4IP B MAC 2 MAC 1 OTV Data Plane: Unicast Core MAC 4 MAC 3 OTV External IP A External IP B West East L2 L3 L2 Animated Slide ! OTV Inter-Site Traffic MAC Table contains MAC addresses reachable through IP addresses OTV Encap 2 Layer 2 Lookup 1 3 Decap 4 MAC 1  MAC 3 6 MAC TABLE VLANMACIF 100MAC 1IP A 100MAC 2IP A 100MAC 3Eth 3 100MAC 4Eth 4 Eth 1 Eth 2 Layer 2 Lookup 5 MAC 1  MAC 3 IP A  IP B MAC 1  MAC 3 IP A  IP B MAC 1  MAC 3 No Pseudo-Wire state is maintained. The encapsulation is done based on a Layer 2 destination lookup. The encapsulation is done in hardware by the Forwarding Engine. No Pseudo-Wire state is maintained. The encapsulation is done based on a Layer 2 destination lookup. The encapsulation is done in hardware by the Forwarding Engine.

15. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 15 Building the MAC tables The OTV Control Plane  The OTV control plane proactively advertises MAC reachability (control- plane learning).  The MAC addresses are advertised in the background once OTV has been configured.  No protocol specific configuration is required. Core IP A IP B IP C West East South MAC Addresses Reachability

16. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 16 OTV Control Plane MAC address advertisements – Multicast Core  Every time an Edge Device learns a new MAC address, the OTV control plane will advertise it together with its associated VLAN IDs and IP next hop.  The IP next hops are the addresses of the Edge Devices through which these MACs are reachable in the core.  A single update reaches all neighbors. Core IP A IP B West East 3 New MACs are learned on VLAN 100 Vlan 100MAC A Vlan 100MAC B Vlan 100MAC C IP C South-East VLANMACIF 100 MAC A IP A 100 MAC B IP A 100 MAC C IP A 4 OTV update is replicated by the core Animated Slide ! OTV Update OTV Update 3 OTV Update 3 2 VLANMACIF 100 MAC A IP A 100 MAC B IP A 100 MAC C IP A 4 3 New MACs are learned on VLAN 100 1

17. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 17 Multicast Groups in the Core  OTV will leverage the multicast capabilities of the core.  This is the summary of the Multicast groups used by OTV:  An ASM/Bidir group to exchange MAC reachability.  An SSM group range for the multicast data generated by the site. Summary of the Multicast groups used by OTV

18. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 18 What if core multicast is not an option? OTV in Unicast Mode – The Adjacency Server Mode  The use of multicast in the core provides significant benefits: Reduce the amount of hellos and updates OTV must issue Streamline neighbor discovery, site adds and removes Optimize the handling of broadcast and multicast data traffic  However multicast may not always be available  The OTV Adjacency Server Mode of operation provides a unicast based solution.

19. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 19 Adjacency Server  Despite the naming this is NOT a physical server. It is just a mode of operation of the Edge Devices.  An OTV node which sends a multicast packet on a non-multicast capable network will “unicast replicate (head-end)” the packet.  One of the OTV Edge Devices will be configured as an Adjacency Server and it will be responsible for communicating the IP addresses where the other Edge Devices can be reached.  The group of IP addresses is called overlay Adjacency List (oAL)  Two configuration steps: 1.Configure an OTV Edge Device to be the Adjacency Server 2.Configure the other Edge Devices to point to the Adjacency Server to retrieve each other IP address Core with no support for multicast: Adjacency Server

20. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 20 Adjacency Server  At first, the Adjacency Server knows about no other OTV Edge Devices because their oAL is empty.  Once other OTV Edge Devices start sending to the Adjacency Servers their site-id and IP address, the Adjacency Server will build up its oAL.  The contents of the oAL is advertised and sent unicast to each member of the oAL. Now the Edge Devices can communicate with each other. IP A Site 1 Site 2 Site 3 Site 4 Site 5 Core IP B IP C IP D IP E Adjacency Server Site2, IP B Site3, IP C Site4, IP D Site5, IP E oAL Site 1, IP A Site 2, IP B Site 3, IP C Site 4, IP D Site 5, IP E

21. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 21 STP BPDU Handling  When STP is configured at a site, an Edge Device will send and receive BPDUs on the internal interfaces.  An OTV Edge Device will not originate or forward BPDUs on the overlay network.  An OTV Edge Device can become (but it is not required to) a root of one or more spanning trees within the site.  An OTV Edge Device will take the typical action when receiving Topology Change Notification (TCNs) messages. OTV Core The BPDUs stop here

22. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 22 Unknown Unicast Packet Handling  Flooding of unknown unicast over the overlay is not required and is therefore suppressed.  Any unknown unicasts that reach the OTV edge device will not be forwarded onto the overlay.  The assumption here is that the end-points connected to the network are not silent or uni-directional.  MAC addresses for uni-directional host are learnt and advertised by snooping the host’s ARP reply OTV Core No MAC 3 in the MAC Table MAC 1  MAC 3 MAC TABLE VLANMACIF 100MAC 1Eth1 100MAC 2IP B

23. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 23 Controlling ARP traffic Proxy ARP  OTV Edge Devices can proxy ARP replies on behalf of remote hosts  ARP traffic spanning multiple sites can thus be significantly reduced  An ARP cache is maintained by every OTV edge device  The ARP cache is populated by snooping ARP replies  Initial ARP requests are broadcasted to all sites  Subsequent ARP requests are suppressed and answered locally  The ARP cache could also be populated at MAC learning time, this would allow the suppression of all ARP related broadcast Core OTV AED OTV ARP Cache MAC 1IP 1 MAC 2IP 2 ARP reply 2 First ARP request (IP A) 1 Snoop & cache ARP reply 3 Subsequent ARP requests (IP A) 4 Proxy ARP reply (IP A) 5 One time traffic

24. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 24 OTV solves Layer 2 fault propagation Summary  STP Isolation: BPDUs are not forwarded over the overlay  Unknown unicasts are not flooded across sites Selective flooding is optional  Cross site ARP traffic is reduced with Proxy ARP  Broadcast can be controlled based on a white list as well as a rate limiting profile

25. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 25 Multi-Homing: Loop Condition Handling  OTV includes the logic necessary to avoid the creation of loops in multi-homed site scenarios.  Each site will have its own STP domain, which is separate and independent from the STP domains in other sites, even though all sites will be part of common Layer 2 domain. Core OTV OTV OTV OTV STP domain 1 STP domain 2 No STP

26. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 26 Authoritative Edge Device  OTV provides loop-free multi-homing by electing a designated forwarding device per site for each VLAN.  The designated forwarder is referred to as the Authoritative Edge Device (AED).  The Edge Devices at the site peer with each other on the internal interfaces to elect the AED  The AED is the only edge device that will forward multicast and broadcast traffic between a site and the overlay. Core OTV OTV OTV OTV AED

27. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 27 Multi-Homing: AED & Broadcast  A broadcast packet gets to all the Edge Devices within a site.  The AED for the VLAN is the only Edge Device that forwards broadcast packets on the overlay network.  All the Edge Devices at a remote site will receive the broadcast packet, but only the AED at the remote site will forward the packet into the site.  Once sent into the site, the packet gets to all switches on the site specific Spanning Tree. Core OTV OTV OTV AED Bcast pkt Broadcast stops here Broadcast stops here OTV

28. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 28 Multi-Homing AED & Unicast Forwarding  One AED is elected for each VLAN on each site  Different AEDs can be elected for each VLAN to balance traffic load  Only the AED forwards unicast traffic to and from the overlay  Only the AED advertises MAC addresses for any given site/VLAN  Unicast routes will point to the AED on the corresponding remote site/VLAN Core OTV OTV OTV OTV AED MAC TABLE VLANMACIF 100MAC 1IP A 200MAC 2IP B IP A IP B

29. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 29 Configuration OTV CLI configuration interface Overlay0 description otv-demo otv join-interface Ethernet1/1 otv control-group 239.1.1.1 otv data-group 232.192.1.2/32 otv extend-vlan 100-150 otv site-vlan 100 Connects to the core. Used to join the Overlay network. Its IP address is used as source IP for the OTV encap ASM/Bidir group in the core used for the OTV Control Plane. SSM group range used to carry the site’s mcast traffic data. Site VLANs being extended by OTV VLAN used within the Site for communication between the site’s Edge Devices

30. © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public BRKRST-2033 30 Configuration OTV CLI configuration with adjacency server interface Overlay0 description otv-demo otv join-interface Ethernet1/1 otv adjacency-server or otv use-adjacency-server 10.10.10.10 otv extend-vlan 100-150 otv site-vlan 100 Connect to the core. Used to join the core mcast groups. Their IP addresses are used as source IP for the OTV encap Configures this Edge device as an Adjacency Server Use a remote Edge Device as the Adjacency Server (mutually exclusive with the previous line) Site VLANs being extended by OTV VLAN used within the Site for communication between the site’s Edge Devices

31. © 2009 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 31 Nexus 7000 Rollout plan  EFT Target start date: Mid January, 2010  FCS Q1CY2010