Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Introduction to Storage Area Networks – I iSCSI & FCIP Ashish Shah Cisco Systems, Inc.

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Agenda iSCSI –What is iSCSI –Login and Session Establishment –Basic IO –Advanced Concepts –Initiators and Targets FCIP –What is FCIP –FCIP Case Study References

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 What is iSCSI RFC Internet Small Computer Systems Interface A SCSI transport protocol that operates on top of TCP –Encapsulates SCSI CDBs (Command Descriptor Blocks) and data into TCP/IP byte-streams –Allows IP hosts to access IP based SCSI targets Broad industry support –Server vendors now publishing own supported iSCSI drivers –Native iSCSI storage arrays now appearing IP IP TCP TCP iSCSI iSCSI SCSI SCSI Data Data

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Architectural Model SCSI Block Commands SCSI Stream Commands Parallel SCSI Transport SCSI Applications (File Systems, Databases) Parallel SCSI Interfaces SCSI Device-Type Commands SCSI Generic Commands SCSI Transport Protocols Layer 3 Network Transport Layer 2 Network Fibre ChannelEthernet, PPP, HDLC… Other SCSI Commands IP TCP SCSI Commands, Data and Status FCP SCSI over FC iSCSI SCSI over TCP/IP

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Conceptual model Network Entity –An object that can have multiple iSCSI nodes iSCSI Node –Initiator or Target node –Identified by an iSCSI Name –iSCSI Target node can have multiple LUs Network Portal –Provides a mechanism to access the iSCSI nodes –Identified by a Network address (IP) Network Portal ( ) Network Portal (IP ) Network Portal (IP ) IP Network iSCSI Initiator Entity iSCSI Target Entity iSCSI node: Iqn.com.cisco.init-1 iSCSI node: Iqn.com.cisco.disk-1 iSCSI node: Iqn.com.cisco.disk-2

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Sessions iSCSI has the concept of a session between an initiator and a target –Loosely equivalent to a SCSI I-T Nexus –It can have one or more TCP connections –Session establishment has three phases Security authentication, Operational negotiation and Full Feature Discovery Sessions –Discover the targets in the network entity –Initiator executes “SendTargets” command after establishing the session This is the only command supported for a Discovery session Target Sessions –Sessions to specific targets –Supports normal SCSI commands

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Discovery Session Login Response Target list: com.cisco.Target-disk1 Com.cisco.Target-Disk2 Session type: Discovery-Session TCP port 3260 Target Initiator iSCSI Initiator Login (Discovery Session) Establish TCP Session TCP iSCSI Driver Command - SendTargets Command - Login Optional Authentication Phase

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Target Session Login TCP port 3260 Target Initiator TCP iSCSI Driver Establish TCP Session Command - Login Session-Type: Normal-Session, Target: com.cisco.Target-disk1 Optional Authentication Phase Operational Parameter Negotiation After this completes we are in full feature mode Command – SCSI commands Normal SCSI commands…

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Common iSCSI Identifiers PGT: Portal Group Tag –Tag identifying a set of Network Portals that can support sessions with multiple connections spanned across these portals ISID: Initiator Session Identifier –Specified by the initiator during login SSID: Session Identifier –Composed of the {ISID, Target PGT} TSIH: Target Session Identifying Handle –A Target assigned tag to identify sessions on its end ITT: Initiator Task Tag –Identifies an iSCSI Task

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Naming iSCSI name can be either : –IQN (iSCSI Qualified Name) iqn => iqn.yyyy-mm.reverse-FQDN-[:|.]-and-qualifier-string For example, iqn com.cisco.cluster1.mbr2 –EUI-64 name (Extended Unique Identifier) eui. EUI-64 For example, eui A425678D

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Packet Format Destination Address Source Address TypeIP 46–1500 Bytes 8662 FCS 4 Octet Preamble TCP Header SCSI Encapsulated Source Port Sequence Number Acknowledgment Number Checksum OffsetReservedU A P R S F iSCSI Targets Will Listen on This Port (by Default) Source PortDestination Port Sequence Number Acknowledgement Number OffsetReservedUAPRSF Window ChecksumUrgent Pointer Options and Padding OpcodeOpcode-Specific Fields Length of Data (After 40-Byte Header) LUN or Opcode-Specific Fields Initiator-Task Tag Opcode-Specific Fields Data Field… TCPData 3260 iSCSI Well-Known Ports: 21 FTP 25 SMTP 80 HTTP 3225 FCIP

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Read Sequence Target Initiator SCSI Command : ITT: 100, CmdSN: 110, ExpStatSN 50 SCSI Data-In: ITT 100, ExpCmdSN: 111, MaxCmdSN: 115, DataSN: 1 SCSI Response: ITT 100, ExpCmdSN: 111, MaxCmdSN: 116, ExpDataSn: 3, StatSN: 50

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Write Sequence Target Initiator SCSI Command : ITT: 100, CmdSN: 110, ExpStatSN 50 SCSI Data-Out: ITT 100, : 111, TTT: 10, DataSN: 1, ExpStatSN: 1 SCSI Response: ITT 100, ExpCmdSN: 111, MaxCmdSN: 116, StatSN: 50 R2T : ITT: 100, TTT: 10 R2TSN: 10

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 More iSCSI Concepts Error Recovery Levels (ERL 0/1/2) –Progressively higher level of error recovery capability 0 => Session failure recovery 1 => Digest failure recovery 2 => Connection recovery iSCSI Header and Data Digests –Integrity protection only –Use of IPSec recommended for security Task Management Functions –Derived from SCSI TMFs

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Initiators NIC AdapterSCSI Adapter iSCSI Software Driver NIC Driver Adapter Driver TCP/IP Stack iSCSI SCSI Generic Block Device Applications File System SCSI Adapter iSCSI Software Driver and Dedicated Hardware Adapter Driver iSCSI TOE Adapter NIC Driver TCP/IP Stack iSCSI SCSI Generic Block Device File System Applications Windows & Linux iSCSI initiators freely available iSCSI HBA: both iSCSI & TCP offloaded to hardware TOE: Only TCP Offloaded

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Targets Native iSCSI Targets just appearing in the market iSCSI Gateway products have been on the market for a longer time –Exports backend FC targets through virtual iSCSI targets –Allows iSCSI initiators to access FC targets –Allows customers to introduce iSCSI initiators to existing FC SANs

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI Gateway on Cisco MDS iSCSI Initiators Presented as FC Initiators FC Targets Presented as iSCSI Targets pWWN a Fibre Channel SAN View pWWN b pWWN c pWWN d FC iSCSI Real iSCSI Initiator IP Addr iqn.initiator.abc IP Network Fibre Channel Fabric FC iSCSI Virtual iSCSI Target IP Addr iqn.target.abc Real FC Target pWWN 12:34…78 fcid 0x iSCSI iqn.abc.123 iqn.def.123 iqn.ghi.456 iqn.jkl.555 iSCSI Storage View IP Virtual FC Initiator pWWN 12:34…56 fcid 0x550001

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 FCIP

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 What is FCIP RFC 3821 – Fibre Channel over TCP/IP Method to encapsulate FC traffic and send over an IP network –Encapsulating TCP connections are called FCIP Tunnels Connects physically separate SANs through E/TE ports over an IP network –Forms a unified SAN in a single Fibre Channel fabric. –FC Fabric and all FC devices are unaware of the presence of the IP Network Motivation is to connect physically remote FC sites to allow remote disk access, tape backup and live mirroring

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 FCIP… Advanced features help throughput: –Write Acceleration Spoofing Transfer Ready locally –Tape Acceleration Spoofing Tape commands locally –Compression

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 FCIP Case Study

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Cisco IT SAN Case Study BEFORE FCIP: Host-Based Replication Source: Cisco Case Study: FCIP and EMC SRDF/A. November 14, 2005 © 2005 Cisco Systems, Inc. All rights reserved.

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Cisco IT SAN Case Study… AFTER FCIP: Array-Based Replication Source: Cisco Case Study: FCIP and EMC SRDF/A. November 14, 2005 © 2005 Cisco Systems, Inc. All rights reserved.

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 iSCSI & FCIP References IETF IP Storage (IPS) Chapter –List of all relevant RFCs, mailing lists and working group details Storage Networking Industry Association (SNIA) –

Introduction to SAN – 1: iSCSI & FCIPBITS Pilani Alumni Association ( 19, 2006 Q & A