FCIP FCIP Introduction

FCIP FCIP Introduction
This lesson introduces FCIP. Starting with a conceptual overview, this lesson covers protocol basics, FCIP network topologies, components, performance issues, and solutions, and finishes with a comparison to other methods of extending SANs over IP networks. Importance A solid understanding of FCIP is necessary for the design and configuration of geographically extended Fibre Channel SANs.

Objectives Upon completion of this lesson you will be able to explain the key concepts of FCIP and identify basic FCIP design principles. Performance Objective Upon completion of this lesson you will be able to explain the key concepts of FCIP and identify basic FCIP design principles. Enabling Objectives Describe the key concepts of FCIP Identify the components of an FCIP implementation List the features and capabilities of Cisco FCIP products Describe common FCIP applications Describe the basic architecture of an FCIP implementation Describe how FCIP transports FC frames across an IP network Describe the impact of flow control in an FCIP implementation Describe FCIP high-availability configurations Identify FCIP design metrics Identify solutions to common FCIP problems

Outline What is FCIP? FCIP Components Cisco FCIP Products
FCIP Applications FCIP Architecture FCIP Transport FCIP Flow Control High-Availability Configurations Design Metrics Common Problems and Solutions FCIP Vendors Prerequisites Curriculum Unit 2, Modules 1 and 2

What is FCIP? Fibre Channel over IP (FCIP) transparently connects FC SAN islands over IP networks Encapsulates and tunnels FC frames through an IP network Uses TCP for flow control and error handling What is FCIP? Objective Describe the key concepts of FCIP Introduction This section describes the key concepts of FCIP. Definition The Fibre Channel over IP (FCIP) protocol transparently connects Fibre Channel (FC) SANs across an IP network. Facts FCIP connects FC SAN islands over IP networks by introducing a network bridging element between the Fibre Channel devices and the IP network. This bridging element maps FC fabric domains to IP addresses and encapsulates FC frames into IP packets. At the receiving end, the network interface element strips off the IP “envelope” and passes the native FC frame to the FC device in its original form and in the order it was sent. FCIP tunnels FC frames through an IP network, using TCP for the flow control and error recovery. Example The preceding diagram shows FCIP gateways used to connect two FC SAN islands across an IP WAN. FC FC FC FC FC FC FC FC IP FCIP FCIP FC switch Gateway Gateway FC switch

What is FCIP? (cont.) iSCSI iFCP FCIP FC Applications Operating System
SCSI-3 Command Set iSCSI SCSI-FCP SCSI-FCP SCSI-FCP Facts FCIP is a draft specification that is in development by the Internet Engineering Task Force (IETF) IP Storage (IPS) Working Group. The specification defines the encapsulation of Fibre Channel frames transported by TCP/IP. The result of the encapsulation is to create a virtual Fibre Channel link that connects Fibre Channel devices and fabric elements across IP networks. The FCIP standard describes: The TCP/IP environment required to support virtual Fibre Channel links and provide the capabilities for tunneling Fibre Channel traffic over an IP-based network A common format and procedure for the measurement and calculation of frame transit time through the IP network Data encryption and data security measures used in FCIP environments, including IP Security (IPsec) and Internet Key Encryption (IKE) The preceding diagram shows the FCIP protocol model. iFCP FC-1 – FC-4 FC-1 – FC-4 TCP TCP TCP IP IP IP iSCSI iFCP FCIP FC

What is FCIP? (cont.) ANSI: IETF IPS Working Group:
FC-SW-2—operation of FC switches; defines E_Ports FC-BB-2—mapping of FC switched networks across a TCP/IP network backbone; defines B_Ports IETF IPS Working Group: Fibre Channel over TCP/IP—specifies requirements for transporting FC frames over TCP/IP FC Frame Encapsulation—defines the common Fibre Channel encapsulation format Facts Four basic specifications define FCIP: The ANSI/T11 FC-SW-2 specification describes the operation and interaction of Fibre Channel switches, including E_Ports and fabric operations. E_Ports are used at both ends of an inter-switch link (ISL). E_Ports forward user traffic and control traffic (Class F Switch Internal Link Services [SW_ILS] frames). FC-SW-2 also defines fabric operations, such as zone merges and Domain ID assignment. The ANSI/T11 FC-BB-2 specification is a mapping that pertains to the extension of Fibre Channel switched networks across a TCP/IP network backbone and defines reference models for E_Port and B_Port operation in an extended SAN environment. The IETF IP Storage (IPS) Working Group maintains the Fibre Channel over TCP/IP specification. This specification defines the FCIP protocol. FCIP defines the TCP/IP requirements for transporting Fibre Channel frames over an IP network. The IETF IPS Working Group FC Frame Encapsulation specification defines the common Fibre Channel encapsulation format.

FCIP Components One fabric IP One fabric IP FC switch Gateway Gateway
Objective Identify the components of an FCIP implementation Introduction This section describes the components of an FCIP implementation. Facts An FCIP link is a point-to-point connection between two FC SANs. FCIP is used to transparently bridge two FC “SAN islands,” creating a single FC SAN fabric. The two SANs that are joined by the FCIP link share a common topology and address space. An FCIP link essentially functions as a “virtual ISL” between two SANs. In terms of network components, FCIP is implemented using a FCIP gateway that connects a Fibre Channel switch to an IP network and encapsulates FC frames into IP packets. The gateway function can be implemented in: A dedicated FCIP bridge A multiprotocol switch or router All other network components are either existing FC hosts, switches and storage, or existing IP network components. Note that FCIP is not defined for use on an Arbitrated Loop. An IP network cannot replace the FC loop connections because there is no provision for encapsulating loop primitive signals for transmission over an IP network. FC FC FC FC FC IP Multiprotocol switch Multiprotocol switch

Cisco FCIP Products MDS 9000 IP Services Module (IPS-8):
Eight 1Gb/s ports (LC-type SFPs) Supports FCIP and iSCSI on same port Up to 3 FCIP tunnels per port 512MB buffer shared between odd-even port pairs Ethernet PortChannel, VSAN, VLAN, VRRP TCP performance enhancements Cisco FCIP Products Objective List the features and capabilities of Cisco FCIP products Introduction This section describes the features and capabilities of the Cisco MDS 9000 Family IP Services Module and the FCIP Port Adapter for Cisco 7200/7400 series routers. Facts The MDS 9000 IP Services Module (IPS-8) provides eight 1Gb/s IP ports and supports full routing capabilities with FC ports in the same MDS chassis. Port are designed to use LC-type SFP transceivers. The IPS-8 module supports: FCIP and iSCSI simultaneously on the same port Up to three FCIP tunnels per port 512MB of buffer space shared between odd-even port pairs Ethernet PortChannels, VSANs, VLANs, and VRRP TCP performance enhancements (RFC 1323, Window Scale, SACK, PMTU, PAWS, TCP Keep-Alive, and RTT) Cisco’s FCIP implementations follows the FC-BB-2 specification, and closely follows the FCIP specification. The IPS-8 does not support jumbo frames, and does not support SLP. IP Services Module MDS9509 MDS9506

Cisco FCIP Products (cont.)
SN Storage Router: Supports FCIP or iSCSI (but not yet on same router) Eight 1–2Gb/s FC Fx/E_Ports Two GbE ports High-availability (HA) clustering VLANs, ACLs, LUN mapping FC zones Facts The Cisco SN Storage Router combines an FC workgroup switch with support for iSCSI or FCIP. Unlike the IPS-8, however, the SN cannot yet support iSCSI and FCIP on the same router. The SN has eight 1–2Gb/s FC ports (capable of F_Port, FL_Port, and E_Port operation) and two GbE ports. The SN supports FC Name Server functions, FC Registered State Change Notification (RSCN) functions, and FC zoning.

FCIP Port Adapter for 7200/7400 (PA-FC-1G): Contains 1 B_Port Up to OC-3 data rates (155Mb/s) supported Supports 7401, 7200 VXR, NPE-G1, NSE-1, NPE-400 VPN acceleration module (SA-VAM) for encryption and compression (2:1) Up to 2 FCIP PAs per router One FCIP tunnel per adapter FCIP Port Adapter Facts The Cisco FCIP Port Adapter for Cisco 7200 and 7400 routers (PA-FC-1G) provides one FC B_Port. The B_Port is connected to an E_Port on an FC switch. The PA-FC-1G supports: Cisco currently supports the PA-FC-1G at data rates up to OC-3 data rate (155Mb/s) VPN acceleration module (SA-VAM) for encryption and compression (2:1) Cisco 7401, 7200 VXR, NPE-G1, NSE-1, and NPE-400 modules Up to 2 FCIP PAs per router One FCIP tunnel per adapter Note that the supported throughput is fairly low for the PA-FC-1G—only 155Mb/s is currently supported. This due in part to the relatively small maximum MWS value of 512KB that is supported by the PA-FC-1G. A 512KB buffer will support 155MB/s at latencies up to about 30–35ms. Cisco 7200 Cisco 7400

PA slot 5 PA slot 6 PCI 0 PCI 1 PA-POS-OC3MM PA slot 3 PA slot 4 SA-VAM PA slot 1 PA slot 2 PA-FC-1G I/O Controller Cisco 7200 with NPE-225, NPE-400, or NSE PA slot 5 PA slot 6 When designing for maximum performance with the PA-FC-1G, it is important to understand the implications of the PCI bus structure of the 7200 series router. On the 7200 series router, there are 2 or 3 PCI buses internal to the chassis: PCI bus 0 controls slot 0 (the I/O controller) and PA slots 1, 3 and 5. PCI bus 1 controls PA slots 2, 4 and 6. With the NPE-G1, there is a third PCI bus for the I/O controller and direct CPU access to the on-board Gigabit Ethernet interfaces. Each PCI bus has a raw throughput of approximately 1.6Gb/s. This effectively limits the PA-FC-1G to about 500Mb/s if there are two other Port Adapters on the same PCI bus. Additionally, when data is fast-switched between Port Adapters, it must move to and from the 7200 system memory over the PCI bus. If the data is transmitted between two port adapters on the same bus, the data must traverse the same PCI bus twice, effectively reducing the performance of the data path by half. Testing on a 7200 with a NPE-400, PA-FC-1G and IO-CTL-GE-E I/O controller shows a 50% decrease in throughput if the PA-FC-1G is installed in an odd (1, 3 or 5) PA slot number shared with the IO-CTL-GE versus when it is installed in an even (2, 4 or 6) PA slot number. To achieve maximum performance, follow these guidelines: Install the FC-PA-1G in an even number slot of the 7200 router (2, 4 or 6). Use the I/O controller interfaces or Port Adapter modules in the the odd number slots (1, 3 or 5) for FCIP traffic. PCI 0 PCI 1 PA slot 3 PA slot 4 PA slot 1 PA slot 2 PA-FC-1G PCI 2 I/O Controller GigE Cisco 7200 with NPE-G1

Certification strategy: Disaster recovery/business continuance focus: EMC SRDF over IP HDS TrueCopy HP/Compaq DRM/CA IBM PPRC/XRC Asynchronous and synchronous Simulation testing: Distance simulation (latency) Varying MTUs Line error conditions Facts Cisco’s certification strategy for the IPS-8 and the PA-FC-1G focuses on working to obtain storage vendor certification for disaster recovery and business continuance solutions, such as: EMC SRDF over IP HDS TrueCopy HP/Compaq DRM/CA IBM PPRC and XRC Although asynchronous data replication is easier to achieve over FCIP, Cisco also plans to certify Cisco FCIP products for synchronous replication given an appropriate WAN environment. To certify FCIP for remote data replication solutions, Cisco works with storage vendors to validate the performance of FCIP products over varying distances (by simulating latency), varying MTU sizes, and typical line error conditions.

FCIP Applications Environments: FC SANs at multiple sites
Need for SAN connectivity between sites Robust IP WAN infrastructure No real-time, synchronous I/O FCIP Applications Objective Describe common FCIP applications Introduction This section describes storage networking situations in which FCIP might be used. Guidelines FCIP might be a valid solution in the following environments: Multiple Fibre Channel SANs exist at multiple geographic locations. There is a need to provide access between FC nodes in two or more remote SANs. There is an existing IP WAN infrastructure that provides sufficient bandwidth and acceptable latency for the customer’s applications. Real-time, synchronous I/O operations are not required.

FCIP Applications (cont.)
Interconnect campus SAN islands Remote backup Remote asynchronous data replication SSPs Content distribution Distributed video production Examples Examples of storage networking applications for FCIP include: Interconnecting SAN “islands” at multiple campuses Remote data backup Remote asynchronous data replication applications, including distributed clustering applications Storage Service Providers (SSPs) using FC within a data storage operations center and using IP for customer access at the SAN level. Content distribution applications that push content across distributed edge networks Distributed video production facilities

Interconnect SAN islands: IPS-8 to IPS-8 IPS-8 to SN IPS-8 to PA-FC-1G SN to PA-FC-1G 7200 VXR w/PA-FC-1G FC FC FC SN FC FC FC The Cisco IPS-8 Port Adapter provides the flexibility to connect remote SAN islands using an IPS-8 at the corporate data center and: FCIP to an IPS-8 at the remote site FCIP to an SN Storage Router at the remote site FCIP to a Cisco 7200 or 7400 router with a PA-FC-1G module at the remote site The IPS-8 can be combined with the the SN and the PA-FC-1G and deployed in a hub-and-spoke topology to optimize deployment cost between the data center and remote sites. FC IPS-8 FC IPS-8 FC FC FC FC FC Remote Sites Corporate HQ

Remote data replication: Wide-area asynchronous replication for disaster recovery Exact copy of server configuration at remote site SRDF (EMC), HOARC (HDS), VVR (Veritas), PPRC (IBM) Manual application failover Application Servers FC FC FC FC Standby Servers The primary type of application for an FCIP implementation is a disk replication application used for business continuance or disaster recovery. Examples of this types of application include: Array-based replication schemes such as EMC Symmetrix Remote Data Facility (SRDF), Hitachi True Copy, IBM Peer-to-Peer Remote Copy (PPRC), or HP/Compaq Data Replication Manager (DRM). Host-based application schemes such as Veritas Volume Replicator (VVR). Replication applications can be run in a synchronous mode, where an acknowledgement of a disk write is not sent until the remote copy is done, or in an asynchronous mode, where disk writes are acknowledged before the remote copy is completed. Applications that are using synchronous copy replication are very sensitive to latency delays and might be subject to unacceptable performance. Customer requirements should be carefully weighed when deploying an FCIP link in a synchronous environment. FCIP is generally more suitable for asynchronous replication. FC FC Mirrored LUNs Asynchronous remote replication Primary Site Recovery Site

Distributed cluster application: Remote-clustered application servers Cluster heartbeat and automated application fail-over Geospan (EMC), Geocluster (Microsoft) Cluster heartbeat Application Servers FC FC FC FC Application Servers Distributed cluster applications can be built on a foundation of data replication. As with basic data replication applications, data is mirrored either synchronously or asynchronously to a remote site. In a distributed cluster, the application servers at both sites are also connected, typically through a data messaging network (such as ATM). This configuration is similar in principle to a basic business continuance/disaster recovery approach, but with a cluster, applications can fail over more gracefully. In addition, the sites can operate in an active-active configuration, where both sites are operational, instead of in an active-standby configuration, where the second site is only used when the primary site fails. FC FC Mirrored LUNs Asynchronous or synchronous remote replication Site 1 Site 2

Remote backup: Data is backed up to remote location Remote copy of data backup purposes NetBackup (Veritas), Celestra Power (Legato) Application Servers FC FC FC Backup Application Another core application for FCIP is remote backup, sometimes known as remote vaulting. In this approach, data is backed up using standard backup applications, such as Veritas NetBackup or Legato Celestra Power, but the backup site is located at a remote location. FCIP is an ideal solution for remote backup applications because: FCIP is relatively inexpensive compared to optical storage networking Enterprises and Storage Service Providers (SSPs) can provide remote vaulting services using existing IP WAN infrastructures Backup applications are sensitive to high latency, but in a properly designed SAN the application can be protected from problems with the backup process by using techniques such as snapshots and split mirrors. FC Tape Library FC Asynchronous remote backup Primary Site Backup Site

B_Port (Bridge) Implementation
FCIP Architecture Node Switch Switch Node N F F N FSPF BF and RCF RSCN Zoning E E ISL ISL GigE FC E E FCIP Architecture Objective Describe the basic architecture of an FCIP implementation Introduction This section describes the basic architecture of FCIP, including port types and encapsulated protocol architecture. Facts The FC-SW-2 and FC-BB-2 specifications define a type of port called a Bridge Port (B_Port). B_Ports are found on bridging devices, such as FCIP bridges. The B_Port on the bridging device connects to an E_Port on a FC switch. B_Ports are used to interconnect switches that do not have built-in ‘backbone’ ports, and are typically implemented in “SAN extender” solutions such as the Cisco FCIP Port Adapter for 7200 and 7400 series routers. Each interconnection is called a FCIP Link and can contain one or more TCP connections. The FCIP Link carries TCP-encapsulated Fibre Channel traffic between Link End Points (LEPs) over an IP network. E_Ports communicate over the IP network using standard FC SW_ILS frames, just like normally interconnected E_Ports communicate over a normal ISL. Supported SW_ILS functions include the Fabric Shortest Path First (FSPF) routing protocol, the Build Fabric (BF) and Reconfigure Fabric (RCF) processes, Registered State Change Notifications (RSCNs), and zoning. The preceding diagram illustrates an FCIP configuration that uses an FCIP bridge at either end of the link. This configuration could be implemented using Cisco 7200 or 7400 series routers with the FCIP port adapter. IP E B LEP FCIP Bridge FCIP Bridge Switch Switch B_Port (Bridge) Implementation (SN and PA-FC-1G)

FCIP Architecture (cont.)
Node Switch Switch Node N F F N FSPF BF and RCF RSCN Zoning FSPF BF and RCF RSCN Zoning Port Channels VSAN trunking E E ISL ISL GigE Logical entity E E When FCIP connectivity is provided by a multiprotocol storage switch, like the MDS 9000 IP Storage Services module (IPS-8), the bridging function is implemented in the switch instead of in a separate device, and B_Ports are not necessary. In this case, the LEP is the Gigabit Ethernet port that resides in the IPS-8. In this implementation, each end of the FCIP Link is associated to a Virtual E_Port (VE_Port), forming a Virtual ISL. VE_Ports communicate over a Virtual ISL using standard FC SW_ILS frames, just like E_Ports communicate over a FCIP bridge configuration. If the VE_Port is configured for trunking, it becomes a TVE_Port. VE_Ports and TVE_Ports behave exactly as E_Ports and TE_Ports. For example: [T]VE_Ports negotiate the same parameters as E_Ports, including Domain ID selection, FSPF, and zones. [T]VE_Ports can be members of a Port Channel. TVE_Ports carry multiple VSANs. The preceding diagram illustrates an FCIP configuration that uses two multiprotocol switches. This configuration could be implemented using the IPS-8 port module. IP LEP VE Virtual ISL Multiprotocol switch Multiprotocol switch VE_Port (Switch) Implementation (IPS-8)

Node Switch Switch Node N F F N FSPF BF and RCF RSCN Zoning E E ISL ISL GigE FC E E When an FCIP link consists of an IPS-8 at one end and an 7200/7400 FCIP port adapter at the other end, the IPS-8 must implement a virtual B_Port to interoperate with the FCIP port adapter. IP B LEP E VB FCIP Bridge Multiprotocol switch Switch VB_Port (Switch-to-Bridge) Implementation

E_Port B_Port FC-SW-2 FC-BB-2 VE_Port B_Access FC-BB-2 FC Entity LEP LEP FCIP Entity FCIP DE DE DE DE DE DE The preceding diagram illustrates the internal architecture of an FCIP link: In an VE_Port implementation, an E_Port connects to a logical VE_Port; in a bridge implementation, a B_Port connects to a logical entity known as a B_Access. VE_Ports and B_Access ports are FC Entities. VE_Port and B_Access entities connect to the FCIP_LEP. The FCIP_LEP is the translation point between an FC Entity and an IP network. The FCIP_LEP coordinates between FC and TCP flow control mechanisms. A LEP can support multiple TCP connections. In the LEP, each TCP connection is managed by a logical entity called an FCIP Data Engine (FCIP_DE). An FCIP Link consists of one or more TCP connections, where each TCP connection connects an FCIP_DE and a TCP Port. The FC-BB-2 specification defines the operation of VE_Ports and B_Ports, while the FC-SW-2 specification defines the operation of E_Ports. The FCIP specification defines the operation of FCIP_LEPs and FCIP Links. FCIP Link TCP TCP Port Port Port Port Port Port Physical Link

The routing / switching element represents an FC switch core Note absence of switch element The preceding diagram shows the internal architecture of an E_Port and a B_Port as defined by the ANSI FC-BB-2 specification. Note that the only difference between the E_Port and B_Port implementations is the absence of a switching element in the B_Port. In a bridge-to-bridge configuration, the E_Ports that are connected to the B_Ports perform the switching functions. Other FC Entity components that are implemented by both the VE_Port and B_Port models include a Control and Services Module and a Platform Management Module that can provide time synchronization, discovery, and security functions. VE_Port B_Port

Different ELS are used by B_Port and VE_Port B_Port The preceding diagram shows a protocol-level view of a B_Port and an E_Port. The architecture of the FCIP Link itself is identical, which permits configurations in which a B_Port is used at one end and a VE_Port is used at the other end. However, B_Ports and VE_Ports are required to support different ELS, so the design of the B_Port and VE_Ports must specifically address compatibility issues. For example, a VE_Port in an MDS might not be compatible with a B_Port from another vendor. All Cisco VE_Port and B_Port implementations now interoperate with each other. FCIP Link is identical VE_Port

Time stamp (seconds fraction)
FCIP Transport Protocol Version ~Protocol ~Version Protocol-specific Flags Length ~Flags ~Length Time stamp (seconds) Time stamp (seconds fraction) CRC 1 2 3 4 5 6 7 8 15 16 23 24 31 Word Bit FC Payload FCIP Transport Objective Describe how FCIP transports FC frames across an IP network Introduction This section explains the process by which FCIP transports FC frames across an IP network. Facts The preceding diagram shows the frame headers defined by the IETF IPS FC Frame Encapsulation specification for an encapsulated FC frame. The header fields specify: Protocol and version Two words for protocol-specific fields Flags Frame length Time stamp CRC The time stamp field is a key component of the header. The sending FCIP Entity sets the time stamp when it injects the frame into the IP network, and the receiving FCIP Entity checks the time stamp when it receives the frame. The frame is dropped if the transit time exceeded FC’s R_A_TOV timeout value. Two mechanisms are provided for validating an FC encapsulation header. Two mechanisms are provided to address the needs of different operating environments. The mechanisms are: Redundancy—the ~Protocol, ~Version, ~Flags, and ~Length fields contain the 1s complement of the Protocol, Version, Flags, and Length fields, respectively. The FCIP receiver can use the redundant fields to validate the contents of the header. CRC—The CRC field is optional. If Bit 5 of the Flags field is set to 1, the CRC is valid. FC IP TCP FCFE SOF EOF CRC CRC Fibre Channel frame

FCIP Transport (cont.) Configuration parameters (can use SLPv2):
IP address and TCP port Destination WWN TCP and QoS connection parameters FCIP Entity generates a TCP connection request to Well Known Port 3225 to open a TCP port for the FCIP Link. First frame transmitted in each direction is an FCIP Special Frame (FSF) Process The following steps occur when an FCIP link is established: Connection parameters needs to be configured before the TCP tunnel can be established. The FCIP specification references Service Location Protocol version 2 (SLPv2) as a possible mechanism to dynamically discover other FCIP Entities. However, early implementations do not use SLPv2, and the following parameters must be manually configured: The IP address and TCP port to which the TCP connection is to be made The WWN of the destination FC Fabric Entity TCP and QoS connection parameters The FCIP Entity generates a TCP connection request to the FCIP Well Known Port (TCP port 3225) at the specified IP address. This action opens a TCP port for the FCIP Link. The first frame transmitted in each direction across the FCIP Link is an FCIP Special Frame (FSF). The FSF is used to identify the peers (FCIP Entities) on either side of the FCIP Link. The FCIP receiver echoes the FSF back to the originator. When the originator receives the response FSF, the tunnel establishment process is complete. Tunnel IP network FC fabric FC fabric FCIP FCIP FSF FSF

FCIP Transport (cont.) FCIP Special Frame (FSF) 7 8 15 16 23 24 31 Bit
7 8 15 16 23 24 31 Bit Protocol Version ~Protocol ~Version Flags Length ~Flags ~Length Time stamp (seconds) Time stamp (seconds fraction) CRC 1 2 3 4 5 6 Word pFlags (Reserved) ~pFlags ~(Reserved) Source FC/FCIP Entity Identifier Connection Nonce FC-BB-2 Usage Code Usage Flags Source FC Fabric World Wide Name Destination FC Fabric World Wide Name K_A_TOV 9 10 11 12 13 14 17 18 FCIP Special Frame (FSF) Facts The preceding diagram shows the fields in a FSF. The type of frame (FSF or normal FC encapsulated frame) is determined by the pFlags field.

Insertion of Time Stamp
FCIP Transport (cont.) After the tunnel has been established, communication is completely transparent to both the nodes and the IP network To comply with FC-FS, the fabric must specify and limit the lifetime of a frame (e.g. R_A_TOV) FC time stamp fields in the encapsulation header are used to calculate the transit time of the frame through the IP network Data frames are dropped after R_A_TOV Encapsulation Insertion of Time Stamp Decapsulation Time Stamp Check Process After the tunnel has been established, communication is completely transparent to both the nodes and the IP network To comply with FC-FS, the fabric must specify and limit the lifetime of a frame. The FC time stamp fields in the encapsulation header are used to calculate the transit time of the frame through the remote (IP) network: When the FCIP Entity encapsulates an FC frame, it inserts a time stamp. When the receiving FCIP Entity decapsulates the frame, it checks the time stamp. If the frame is delayed too much with respect to the FC timers the frame must be dropped on the remote side of the FCIP Link (that is, not injected into the remote fabric). For example, the R_A_TOV value is used to delay retransmitting a failed FC sequence. R_A_TOV prevents the receiver from confusing frames from the failed sequence with frames from the retransmitted sequence. If the IP network holds a frame for longer than R_A_TOV, frames from a failed sequence might be injected into the fabric after R_A_TOV has expired and the node has begun to retransmit the sequence. For this reason, data frames are always dropped if the transit time across the IP network exceeds R_A_TOV. Class F frames, however, could be retained if doing so was advantageous. Tunnel IP network FC fabric FC fabric FC FCIP FCIP FC

FCIP Flow Control Flow control:
FC BB_Credits between E_Ports and B_Ports TCP sliding window over IP link FCIP Flow Control Objective Describe the impact of flow control in an FCIP implementation Introduction This section describes how FC and TCP flow control techniques are used on an FCIP link. Facts Two types of flow control are used in an FCIP implementation: FC buffer-to-buffer credits (BB_Credits) are used only between the FC switch E_Ports and the FCIP gateway B_Ports on the FC sides of the link. TCP sliding window flow control is used on the IP link BB_Credit TCP sliding window BB_Credit IP E B LEP LEP B E Gateway Gateway Switch Switch

FCIP Flow Control (cont.)
FC buffers are emptied almost immediately as long as there is sufficient IP buffer space BB_Credits have no significant impact on the utilization of the IP link On the FC side of the gateway, the B_Port ASIC receives a frame into its buffer and passes the frame over to the IP side for encapsulation. At this point, the FC buffer is empty and an R_RDY is sent back to the originating E_Port. In other words, FC buffers are emptied almost immediately as long as there is sufficient buffer space on the IP side. Therefore, FC BB_Credits have no significant impact on the utilization of the IP link, and the latency in the IP network does not determine the number of buffer-to-buffer credits required. IP Frame E Frame B LEP R_RDY Switch Gateway

High-Availability Configurations
Use multiple FCIP tunnels for redundancy and increased performance: FSPF load balancing for tunnels on different switches VRRP for tunnels on different switches Ethernet PortChannels for links on the same switch FC PortChannels between two MDS switches High-Availability Configurations Objective Describe FCIP high-availability configurations Introduction This section describes the high-availability configurations that are supported by Cisco FCIP products. Facts Multiple FCIP tunnels can be implemented to enable high-availability configurations and to aggregate bandwidth for increased performance. Cisco FCIP products support the following high-availability configurations: FC Fabric Shortest Path First (FSPF) provides redundancy and load balancing for tunnels on different switches. Virtual Router Redundancy Protocol (VRRP) provides failover for tunnels on different switches. Ethernet PortChannels provide redundant links and bandwidth aggregation for links that are attached to the same switch. FC PortChannels provide redundant paths and bandwidth aggregation for tunnels between the same pair of switches.

High-Availability Configurations (cont.)
FCIP tunnel IP Network FC fabric FSPF load balancing FCIP tunnel The preceding diagram shows a redundant FCIP configuration that uses FSPF load balancing to provide multiple paths. This configuration protects against the failure of a switch or a failure of an IP link. Traffic can be manually load-balanced across both links by configuring FSPF link costs to provide equal cost paths through the fabric to the gateway switches. FSPF is an FC feature, and is supported only by the IPS-8 in an MDS switch. FSPF load balancing (IPS-8 only): Across tunnels on different switches Handles failure of switch or IP link failure Manual load-balancing by setting FSPF link costs

FCIP tunnel IP Network FC fabric FC fabric FCIP tunnel VRRP Group The preceding diagram shows a redundant FCIP configuration that uses VRRP to provide a standby FCIP data path. Two ports on different switches are placed into one VRRP group, which is assigned a single virtual IP address. When the active port fails, the virtual IP address fails over to the standby port and the standby port takes over for the failed port. This configuration protects against the failure of a switch or a failure of an IP link, but does not provide load-balancing. VRRP is supported by the IPS-8 and by the PA-FC-1G FCIP Port Adapter. VRRP failover configuration: Two ports are in one VRRP group with one VRRP IP address When active VRRP port fails, the standby takes over Peer reconnects to same IP address and the link is back up

Ethernet PortChannel FCIP tunnel IP Network FC fabric FC fabric FCIP tunnel The preceding diagram shows a redundant FCIP configuration that uses Ethernet PortChannel to provide redundant links between an IPS-8 module and an attached Ethernet switch. This configuration provides transparent failover between links in the Ethernet PortChannel. However, an Ethernet PortChannel does not support load-balancing or bandwidth aggregation. All traffic on the FCIP tunnel is carried on one TCP connection that traverses one link. Only two Gigabit Ethernet ports can be aggregated, and the ports must be contiguous, odd–even pairs, such as 2/1 and 2/2, or 2/5 and 2/6. Therefore, Ethernet PortChannels offer limited benefits. When there is an IPS-8 at each end of the connection, FC PortChannels can be used instead of Ethernet PortChannels. The VE_Ports on the IPS-8 are virtual TE_Ports, and support FC PortChannel functionality. Using FC PortChannels on the IPS-8 provides redundancy, aggregation, and load-balancing. In addition, each link in an FC PortChannel can be an Ethernet PortChannel. In other words, you can take four IPS-8 blades, with 32 GigE ports, and put them into one FC PortChannel. Only 16 of the links in the PortChannel will be active. Ethernet PortChannel is useful when you are connecting an IPS-8 to an PA-FC-1G FCIP Port Adapter, because the PA does not support FC PortChannels. However, when you are connecting an IPS-8 to an IPS-8, use FC PortChannels instead. Ethernet PortChannel configuration: Ethernet link-level redundancy Supports 2 contiguous GigE ports (odd/even pairs) Does not support aggregation or load-balancing Use when connecting an IPS-8 to a PA-FC-1G

PortChannel FCIP tunnel IP Network FC fabric FCIP tunnel The preceding diagram shows a redundant FCIP configuration that uses the FC PortChannel feature to provide redundant datapaths between two IPS-8 modules at either end of the datapath. This configuration provides transparent failover between links in the PortChannel, and provides bandwidth aggregation across the tunnels. FC PortChannels can be extended across the IP network. PortChannels are an FC feature, and are supported only by the IPS-8 in an MDS switch. FC PortChannel configuration (IPS-8 only): Redundancy across the FCIP tunnel Transparent failover and bandwidth aggregation Use when connecting an IPS-8 to an IPS-8

Short distance <= 60km Medium distance <= 160km
FCIP Design Metrics GigE, OC48 or higher Relatively low latency Suitable for some synchronous apps Campus Ethernet FCIP FCIP Short distance <= 60km OC3 / OC12 Relatively low latency Suitable for some synchronous apps Mainly asynchronous SONET/SDH FCIP FCIP FCIP Design Metrics Objective Identify FCIP design metrics Introduction This section identifies FCIP design metrics. Facts The IP WAN environment is a factor when considering the applicability of FCIP for a specific application. The primary factors are the bandwidth, distance, and latency of the WAN. For example: A campus or metro-area Gigabit Ethernet or short-haul, high-bandwidth SONET/SDH implementation is suitable for some synchronous applications because it provides the bandwidth and low latency required by synchronous replication schemes. A medium-haul OC3 or OC12 SONET/SDH implementation provides sufficient bandwidth and latency for some synchronous applications, but is better suited for asynchronous applications like asynchronous mirroring or remote backup. On a long-haul IP-routed WAN link, such as a DS1 or DS3 link, bandwidth is too low and latency is too high to support most synchronous applications, and might be too low to support asynchronous mirroring. This type of environment is more suitable for applications like remote backup. Medium distance <= 160km Low-speed DS1 / DS3 Higher latency Longer distance Asynchronous IP Routed WAN FCIP FCIP Long distance > 160km

FCIP Design Metrics (cont.)
VSAN 10 FC VSAN 15 FC VSAN 20 IP Network FC FC FC FC Facts The devices that need to communicate over the FCIP link should generally be placed into a separate VSAN. Using VSANs to isolate the FCIP link from the rest of the SAN is recommended for the following reasons: Minimizing the number of devices that can access the FCIP link reduces the amount of fabric traffic on the link. Fabric traffic includes Registered State Change Notifications (RSCNs), inter-switch communication that occurs when a switch or interswitch link (ISL) is brought online, and target probes sent by initiators. Two fabrics joined by an FCIP link form a single contiguous fabric. If the FCIP link goes down, the fabric will split. This results in a FC Build Fabric (BF) event in one of the fabrics (to select a new Principal Switch) and FSPF recomputations in both fabrics as the fabrics adjust to the split. The use of VSANs limits the extent of the disruption to only those ports that need to access the FCIP link. The use of VSANs can also reduce the time it takes for the fabric to reconverge when the FCIP link is restored. If multiple sets of devices communicate over the FCIP link, zones can be used to further reduce RSCNs within the FCIP VSAN. Zones can also be used instead of VSANs if there is a PA-FC-1G or SN at one end of the FCIP link. Example The preceding diagram shows an FCIP implementation that supports an array-to-array mirroring application such as EMC’s SRDF or IBM’s PPRC. A VSAN (VSAN 15) is used to isolate the devices that need to communicate on the FCIP link. Note that the storage arrays belong to two VSANs—each switch port can belong to only one VSAN, so the configuration shown here requires that each array has at least two ports. A dedicated port is typically recommended or required for array-based mirroring solutions. Use VSANs to isolate the FCIP link: Reduces fabric traffic (RSCNs, target probes) Isolates effects of broken FCIP link Zones can be used to further reduce RSCNs within the VSAN, or if one end of the link is a 3rd-party switch

Common Problems and Solutions
FC Error Detect Time-Out Value (E_D_TOV) timeout value might need to be increased due to IP network latency: Default value is 2 seconds Must accommodate in-order delivery of data (and ACKS, if Class 2 is used) across WAN E_D_TOV must be the same for all switches and VSANs Repeated link timeouts indicate that E_D_TOV is too low Common Problems and Solutions Objective Identify solutions to common FCIP problems Introduction This section identifies solutions to common FCIP problems. Facts Because IP latency tends to be large and variable (relative to FC), the FC Error Detect Time-Out Value (E_D_TOV) timeout value might need to be increased. The default value is 2 seconds, which must accommodate in-order delivery of data (and acknowledgement, if Class 2 is used) across the fabric and the IP WAN. For networks with high latency, E_D_TOV might need to be increased. Repeated link timeouts might indicate that the E_D_TOV is too low. Note that E_D_TOV must be the same for all FC switches in the SAN. E_D_TOV must also be the same for all VSANs.

FCIP Vendors Cisco CNT Lucent SANcastle Sun FCIP Vendors Objective
List the key vendors in the FCIP market Introduction This section identifies the key vendors in the FCIP market. Facts The following vendors offer FCIP products: Cisco CNT Lucent SANcastle Sun

Lesson Review Which FCIP implementation is used by the IPS-8?
Which security features are available in an FCIP implementation? What are the key concepts that define FCIP? Practice Which FCIP implementation(s) are used by the IPS-8? a. B_Port b. E_Port c. VB_Port d. VE_Port Which security features are available in an FCIP implementation? a. Fabric zoning b. VSAN c. Internet Key Encryption d. All of the above 3. What are the key concepts that define FCIP? a. Encapsulation of Fibre Channel frames Use of TCP for flow control and Error recovery b. Encapsulation of Fibre Channel frames Uses UDP for flow control and propagates errors to the application layer c. Encapsulation of Fibre Channel sequences Uses TCP for flow control and propagates errors to the application layer

Lesson Review (cont.) Can you connect SAN A to SAN B using FCIP? Why or why not? FC Switch FC Hub FC FC Can you connect SAN A to SAN B using FCIP? Why or why not? FC FC FC FC SAN A SAN B

Lesson Review (cont.) What kind of frames do VE_Ports use to communicate over an FCIP Link? What base specifications define FCIP? What are iSCSI and FCIP primarily used for? What kind of frames do VE_Ports use to communicate over an FCIP Link? a. V_ISL frames b. SW_ILS frames c. E_ISL frames What base specifications define FCIP? a. FC-SW-3, FC-BB-3, FCIP, and FC Frame Encapsulation b. FC-SW-2, FC-BB-2, FCIP, and FC Frame Encapsulation c. FC-SW-2, FC-BB-2, FCIP, and FC Segment Encapsulation What are iSCSI and FCIP primarily used for? (Choose two) a. FCIP is primarily used to provide business continuity over the MAN and WAN. b. FCIP primarily supports host-to-SAN applications such as remote data replication. c. iSCSI is primarily used in small office environments where there is no existing FC SAN. d. iSCSI is primarily used to lower storage TCO by leveraging the existing IP infrastructure and skills. e. iSCSI is primarily used to communicate between multi-protocol switches and FC storage devices

Summary FCIP is a protocol for bridging FC SANs over IP networks
FCIP specifications include FC-SW-2, FC-BB-2, FCIP, and FC Frame Encapsulation FCIP is typically used to connect geographically remote SANs when real-time, synchronous I/O is not required Cisco FCIP products: MDS 9000 IP Services Module (IPS-8) SN Storage Router FCIP Port Adapter for 7200/7400 routers (PA-FC-1G) Summary: FCIP In this lesson, you learned what the FCIP protocol is, and how it works to bridge Fibre Channel and IP networks. You learned that FCIP implementations are subject to the performance limitations of IP networks, and that measures designed to enhance security, performance, and flow control are available for FCIP.

Summary (cont.) FCIP topologies feature Fibre Channel components within each SAN, and gateways that are either: FCIP bridges (B_Port) Multiprotocol storage switches or routers (VE_Port) FCIP encapsulates FC frames with TCP, IP, and FCFE headers, and a CRC, and establishes a tunnel through the IP network to send the encapsulated frames FCIP uses two flow control mechanisms: Buffer credits are used between the B-Ports and E_Ports on the FC sides of the link TCP sliding window is used on the IP link

Summary (cont.) Might need to increase E_D_TOV FCIP HA configurations:
FSPF load balancing VRRP Failover Ethernet PortChannel FC PortChannel FCIP design metrics: Network latency and distance Synchronous vs. asynchronous applications Use of VSANs to isolate FCIP links Might need to increase E_D_TOV

FCIP FCIP Introduction

Similar presentations

Presentation on theme: "FCIP FCIP Introduction"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

FCIP FCIP Introduction

Similar presentations

Presentation on theme: "FCIP FCIP Introduction"— Presentation transcript:

Similar presentations

About project

Feedback