FC Initialization Flow Host (Physical Machine) initiates Fabric Login with nearest switch – FLOGIN – receives a VN_Port ID in return (Additional entities on same Host – e.g. Virtual Machines may also log into fabric using NPIV FDISC, each getting a unique VN_Port ID). (Host may use well-known FC servers to discover the VN_Port IDs of its Targets and their LUNs). Host initiates Port Login with chosen Targets
Ethernet Routing Dynamic Scheme: Source Learning If unicast DstMAC is not in lookup table, flood frame to all ports except its source port. Note source port of SrcMAC in lookup table, if not already present Age/invalidate lookup entries Similar flooding behavior for multicast Precludes loops in fabric
FC Classes of Service Class 1 – dedicated connection – obselete; unsupported in FCoE Class 2 – acknowledged, with ACKs from remote peer. Supported in FCoE, though some link-layer features (F_RJT, F_BSY) irrelevant. Useful for large Sequences and sequential devices. Class 3 – unacknowledged, FCoE-supported. Most prevalent for disk applications. Error recovery punted to higher (FCP+) levels. Class F – inter-switch link only. FCoE-supported
iSCSI SAN Pros and Cons For: Runs over existing, ubiquitous Ethernet, TCP/IP fabrics. Internetworking built in. Lots of fabric management tools (ready for enterprise storage?). Against TCP Slow Start impacts I/O latency, throughput (but newer TCPs are tunable) Lossy fabrics impact I/O latency through re- transmission and complicate receive endpoint data placement. Bridging to legacy FC SANs slow/expensive because of TCP termination overhead.
Converged Ethernet AKA Data Center Bridging (DCB). Run up to four major traffic classes on single 10 GbE fabric. In order of market prevalence: Networking (TCP/IP, lossy). Block Storage (lossless FCoE, or lossless/lossy iSCSI). Management (heartbeat traffic, low bandwidth, but must get through). Inter-Process Communication (clustered computing: high bandwidth, low latency, lossless preferred).
Groundwork for DCB IEEE 802.1Qaz – ETS & DCBX – bandwidth allocation to major traffic classes (Priority Groups); plus DCB management protocol. IEEE 802.1Qbb – Priority PAUSE. Selectively PAUSE traffic on link by Priority Group. IEEE 802.1Qau – Dynamic Congestion Notification.
IEEE 802.1Qaz Enhanced Transmission Selection Support at least 3 Priority Groups/traffic classes PGs identified by Priority field of existing 802.1Q VLAN Tag Configured Bandwidth per PG has 1% resolution PG15 has limitless bandwidth (use sparingly!, for Management) Work Conservation – if the wires free, use it.
ETS Configuration Example PG0 (Storage): 40% of port b/w PG1 (Networking): 20% of port b/w PG2 (IPC): 40% of port b/w PG15 (mgmt): limitless If a PG underutilizes, others can fill the space. Typical implementation: DWRR.
IEEE 802.1Qau Dynamic Congestion Control Background Lossless fabrics are prone to congestion spreading (congestion trees). Ethernet-FC gateways with their different port speeds (10 GbE; 8 Gbps) are natural bottlenecks. ETS Work Conservation model adds fuel to fire. Solution: switches/endpoints notify traffic sources of incipient congestion, via feedback messages; sources reduce rates accordingly.
FCoE Summary Presents new, but very familiar, PHY and Link Layers for FC. Core switching discipline remains FC-SW-5. Higher FC layers almost completely unchanged (thats the legacy value!) Biggest Ethernet-level requirement: lossless fabric. Part of Converged Ethernet initiative – lots of ancillary activity at IEEE.