Presentation on theme: "FCoE Overview IEEE CommSoc/SP Chapter Austin, Texas, May 21 2009 Tony Hurson"— Presentation transcript:
FCoE Overview IEEE CommSoc/SP Chapter Austin, Texas, May Tony Hurson
Networked Storage History
SANs Latency, Throughput Requirement
SCSI Read, Write over FC
FC Fabric Port Terminology
FC Initialization Flow Host (Physical Machine) initiates Fabric Login with nearest switch – FLOGIN – receives a VN_Port ID in return (Additional entities on same Host – e.g. Virtual Machines may also log into fabric using NPIV FDISC, each getting a unique VN_Port ID). (Host may use well-known FC servers to discover the VN_Port IDs of its Targets and their LUNs). Host initiates Port Login with chosen Targets
Ethernet Routing Dynamic Scheme: Source Learning If unicast DstMAC is not in lookup table, flood frame to all ports except its source port. Note source port of SrcMAC in lookup table, if not already present Age/invalidate lookup entries Similar flooding behavior for multicast Precludes loops in fabric
FC Frame Format
FC Classes of Service Class 1 – dedicated connection – obselete; unsupported in FCoE Class 2 – acknowledged, with ACKs from remote peer. Supported in FCoE, though some link-layer features (F_RJT, F_BSY) irrelevant. Useful for large Sequences and sequential devices. Class 3 – unacknowledged, FCoE-supported. Most prevalent for disk applications. Error recovery punted to higher (FCP+) levels. Class F – inter-switch link only. FCoE-supported
Protocol Stack History and Comparison
iSCSI SAN Pros and Cons For: Runs over existing, ubiquitous Ethernet, TCP/IP fabrics. Internetworking built in. Lots of fabric management tools (ready for enterprise storage?). Against TCP Slow Start impacts I/O latency, throughput (but newer TCPs are tunable) Lossy fabrics impact I/O latency through re- transmission and complicate receive endpoint data placement. Bridging to legacy FC SANs slow/expensive because of TCP termination overhead.
Lossless Ethernet – via PAUSE
FCoE Early Deployment Example
FCoE Frame Format
FCoE Endpoint Model
FCoE Switch Functional Model
Converged Ethernet AKA Data Center Bridging (DCB). Run up to four major traffic classes on single 10 GbE fabric. In order of market prevalence: Networking (TCP/IP, lossy). Block Storage (lossless FCoE, or lossless/lossy iSCSI). Management (heartbeat traffic, low bandwidth, but must get through). Inter-Process Communication (clustered computing: high bandwidth, low latency, lossless preferred).
Groundwork for DCB IEEE 802.1Qaz – ETS & DCBX – bandwidth allocation to major traffic classes (Priority Groups); plus DCB management protocol. IEEE 802.1Qbb – Priority PAUSE. Selectively PAUSE traffic on link by Priority Group. IEEE 802.1Qau – Dynamic Congestion Notification.
IEEE 802.1Qaz Enhanced Transmission Selection Support at least 3 Priority Groups/traffic classes PGs identified by Priority field of existing 802.1Q VLAN Tag Configured Bandwidth per PG has 1% resolution PG15 has limitless bandwidth (use sparingly!, for Management) Work Conservation – if the wires free, use it.
ETS Configuration Example PG0 (Storage): 40% of port b/w PG1 (Networking): 20% of port b/w PG2 (IPC): 40% of port b/w PG15 (mgmt): limitless If a PG underutilizes, others can fill the space. Typical implementation: DWRR.
IEEE 802.1Qbb Priority PAUSE
IEEE 802.1Qau Dynamic Congestion Control Background Lossless fabrics are prone to congestion spreading (congestion trees). Ethernet-FC gateways with their different port speeds (10 GbE; 8 Gbps) are natural bottlenecks. ETS Work Conservation model adds fuel to fire. Solution: switches/endpoints notify traffic sources of incipient congestion, via feedback messages; sources reduce rates accordingly.
Congestion Notification in Action
Congestion Control at Endpoint Transmit
FCoE Summary Presents new, but very familiar, PHY and Link Layers for FC. Core switching discipline remains FC-SW-5. Higher FC layers almost completely unchanged (thats the legacy value!) Biggest Ethernet-level requirement: lossless fabric. Part of Converged Ethernet initiative – lots of ancillary activity at IEEE.
Further Reading FCoE: IEEE 802.1Q(az|au|bb): Thank you! Questions?