iSER on SCTP & IB draft-hufferd-ips-iser-sctp-ib-00.txt Generalizations to iSER specification John Hufferd Mike Ko Yaron Haviv

2 Abstract The iSCSI Extensions for RDMA document currently specifies the RDMA data transfer capability for iSCSI over iWARP/TCP This new document generalizes the iSER document to permit it to be used with other RDMA capable protocols such as –iWARP/SCTP –InfiniBand –etc. It also describes what should be defined in the InfiniBand Trade Association and what things are appropriate for specification in the IETF

Motivation Current wordage in iSER is only applicable to iSER on iWARP/TCP It was felt that the protocol should be made generic for RDMA LLPs, and that includes: –iWARP/SCTP –InfiniBand –Etc. Makes an iSCSI based protocol apply across more Networks, and eliminates some of the management and discovery protocols that would otherwise be needed –For example, would not have to fix the short comings of SRP (SCSI RDMA Protocol) on IB

Goals To specify changes/adjustments that should be considered to the wordage in the iSER document to make it more General These changes should not modify the basic operation of iSCSI/iSER when operating on iWARP/TCP Some of the terminology needed to be clarified as to the applicability of the terms to the actual LLP used

Examples of changes (1) The term “iWARP protocol suite” is replaced by “RDMA-Capable Protocol” The term “iWARP layer” will be replaced by “RDMA-capable protocol layer” Wherever the “iWARP” term is specific to the TCP implementation, it will be replaced with “iWARP/TCP” The term “RNIC” will be replaced with “RDMA- Capable Controller” –The clause “such as an RNIC” will be added as needed

Examples of changes (2) The Steering Tag (STag) term will have its definition extended such at the IB “Local Steering Tag (L-Key)” and the “Remote Steering Tag (R-Key)” are included in the STag definition by way of example The Definitions for IRD and ORD terms will have their definition extended such that the IB “Responder Resources”, and Initiator Depth” are included in the definitions by way of example The Term RDMA-capable protocol (RCP) will be defined and used when ever any RDMA wire protocol or RDMA protocol stack is applicable –RDMAP should be used only when it explicitly refers the iWARP protocol (TCP or SCTP)

Examples of changes (3) The Term “RDMAP Stream” will be replaced by the term “RCP Stream” and defined as: –RCP Stream - A single bidirectional association between the peer RDMA-capable protocol layers on two Nodes over a single transport-level stream. For TCP or SCTP, an RCP Stream is also known as an RDMAP Stream. For iSER/TCP, the association is created when the connection transitions to iSER-assisted mode following a successful Login Phase during which iSER support is negotiated. –Needed since SCTP and IB start their RCP stream mode at connection time

Examples of changes (4) The term “RCP Message” should be defined and used as a replacement for the term “RDMAP Message” the definition will be: –RCP Message – The sequence of packets of the RDMA-capable protocol which represent a single RDMA operation or a part of RDMA Read Operation. For TCP or SCTP, an RCP Message is also known as an RDMAP Message When discussing the iSER Hello and HelloReply Messages the term "iSER Message" will be used –Instead of “RDMAP Message” –This distinction is needed in order to accommodate LLPs that have native message delivery capability, such as SCTP or IB

Example of Changes (5) We permit the iSCSI layer (if appropriate) to use the RCP message mode capability immediately after connection establishment before enabling iSER-assisted mode –Appropriate for SCTP or IB –In this case the iSER Hello and HelloReply Messages are not the first RCP Messages, but they are the first iSER Messages Added a discussion of connection establishment along with the use of the “RCP messaging protocol”, for exchanging Login Request and Login Response Messages –Examples are discussed that are appropriate for SCTP and IB, along with the transitioning of the connection to iSER mode

Examples of changes (6) The discussion of Security specifies that all non IP protocols will define their own requirements for IPsec –However the iSCSI requirements for IPsec are still required: Wherever an iSER Message enters an IP environment from a non IP one (such as IB) –The iSCSI/iSER requirement for IPsec on IP based protocols such as TCP and SCTP Will continue to require IPsec as a must implement

Generic example of iSCSI/iSER layering in Full Feature Mode

IB Informative Section Information on how iSER would be used in an IB network is included in the Draft –Not intended to be included in the iSER draft unless included in an informational Appendix –Various Network topologies are shown Host Side IB Network, including Gateways Storage Side that includes iSER/IB –IB iSER discovery process is described Use of IP over IB (IPoIB) –SendTargets –SLP –iSNS Conversion of IP address into IB GID via ARP processes –Discussion of what the IBTA needs to define

Host Side iSER/IB Topology

Storage Side with iSER/IB

Info: What the IBTA needs to define Means for permitting a Host to establish an iSCSI/iSER connection with a peer InfiniBand end-node –Indicating when that end node does not support iSER So the Host would be able to fall back to iSCSI/TCP over IPoIB Means for permitting the Host to establish connections with –IB iSER connections on Storage Controllers or –IB iSER connected Gateways in preference to IPoIB connected Gateways/Bridges or –Connections to Target Storage Controllers that accept iSCSI via IPoIB How to operate in an environment where ZBTO, and SendInvSE are optional iSER ServiceID How the ServiceID can be added to the IP port number during the connection process

Info: Key Implementation areas that do NOT require IBTA specification How implementations determine which iSCSI/iSER portal group to use –Basing the decision on new information that may be placed in the iSCSI discovery information is Not required –Simple trial selection is acceptable How implementations determine how to best handle the concept of MC/S as it deals with multiple IB Addresses:Ports per Portal Group

Preferred Change to Discovery Data Useful to have a connection type associated with the Portal Group Tag –Will permit the most appropriate connections to be made without needless connection tries and failures –Useful for iSER/iWARP/TCP –Useful for iSER/iWARP/SCTP –Useful for iSER/IB –Proposed Syntax IP Address:Port[, PG#[, Type#]] Portal Group Type IANA Portal Group Type Value iSCSI 0, or blank iSER/iWARP/TCP 1 iSER/iWARP/SCTP 2 iSER/iWARP/IB 3

Recommendations Have Mike Ko update the iSER wordage as indicated by this draft Include the IB sections as Informational Appendices in iSER draft Send Discovery Data changes to the authors of iSCSI, iSNS, & SLP for iSCSI