1. Hosted by IP Storage: Best Practices Marc Staimer, President & CDS Dragon Slayer Consulting marcstaimer@earthlink.net 503-579-3763

2. Hosted by Agenda IP Storage Level Setting File vs. Block Storage FCIP, iFCP & iSCSI Fiction & Facts about iSCSI Storage Replication over WANs Considerations for Designing IP Storage Networks Questions

3. Hosted by IP Storage Level Setting There are 3 types: Block Storage  iSCSI File Storage  NAS Storage over WAN for business continuity = Block  iSCSI  FCIP  iFCP

4. Hosted by IP Storage Level Setting iSCSI Storage Block-based external storage on Ethernet  Vs. SCSI, USB, 1394, or FC NAS or Network Attached Storage File-based storage = NFS & CIFS No different than any other file server Requires block storage behind it

5. Hosted by By show of hands, what is acceptable packet loss for IP Block Storage on a LAN/WAN? 1)1% 2)5% 3)10% 4)0%

6. Hosted by Acceptable Packet Loss for IP Block Storage on a LAN/WAN The Answer is:

7. Hosted by Block vs. File Storage IP block data is unlike any other IP data Overwhelms most current LAN/WAN environments Incredible amounts of traffic Tolerates “ZERO” packet loss Very low latency File storage = specialized file server NFS & CIFS Higher prioritization is required depending on app Volume of data may overwhelm untuned LAN/WAN

8. Hosted by FCIP, iFCP, iSCSI FCIP Fibre Channel tunneled in TCP/IP IP transport between FC switches iFCP IP header put on Fibre Channel frames for routing IP connection services for FC devices iSCSI SCSI-3 mapped to TCP

9. Hosted by FCIP Pt-to-Pt: Becomes “One” FC SAN Disruptions pass SAN-to-SAN Large FSPF database PSS between SAN sites Gateways between fabrics (blades or boxes) Ethernet LAN/WAN Switches Cisco MDS 9216 CNT UESR 3000 Brocade 3xxx SAN 1 SAN 2 SAN 3 SAN 4 TCP/IP WAN

10. Hosted by iFCP Pt-to-Multi-Pt Device specific passing only the data that is required Devices can appear in multiple individual SANs  The SANs themselves remain independent Ethernet LAN/WAN Switches SAN 1 SAN 2 SAN 3 SAN 4 TCP/IP WAN McDATA/Nishan 3300

11. Hosted by Parable

12. Hosted by

13. iSCSI: Ethernet-Based SAN The Hype Block storage on Ethernet Leverage current infrastructure investment & knowledge base Lowers cost Eliminates headaches Ubiquitous Makes FC another Ethernet Road Kill

14. Hosted by iSCSI Defined IETF standard protocol Establishes & manages connections Carries storage (SCSI) blocks  From initiators to storage targets Encapsulates SCSI blocks in TCP/IP Then tunneled in Ethernet iSCSI is to Ethernet as FCP is to Fibre Channel Network application One infrastructure for LANs, NAS, & SANs Ethernet Frame TCP/IP Packet SCSI-3 TCP/IP iSCSI NAS: File Storage GbE Switch Mission Critical IA App Servers

15. Hosted by iSCSI Applications NAS/SAN combined storage units Entry level SANs Limited budget SANs

16. Hosted by SAN Benefit Assertions of iSCSI Reduced costs Professional services, implementation, management & IT Staff time Reduced complexity Reduced Management Increased Interoperability Elimination of Multiple Networks Unlimited SAN Distance Equal or Better Performance

17. Hosted by Fiction & Facts about iSCSI Fiction Cost is lots < Fibre Channel (FC) Complexity is much < than FC Uses current infrastructure Requires no storage knowledge Is as fast as FC Will replace FC in Enterprise Easier to manage than FC Eliminates SAN distance limits  Latency (delay) is not an issue Facts Known technology Costs are relatively < FC  Cycles or hardware Doesn’t require special HW  But benefits from it Latency (Delay) matters  Can’t be > 1 millisecond Deterministic routing Doesn’t require any-to-any

18. Hosted by Rating the iSCSI Value Props Reduces or Eliminates SAN Professional Services Lowers SAN Hardware Costs Simplifies SAN Management Eliminates Interoperability Issues Converges SAN/LAN/MAN/WAN Fabric Infrastructure Extends SANs over unlimited distances Equal or better performance than FC SANs

19. Hosted by iSCSI Reality Check There are some real cost benefits for: NAS/SAN on the same fabric infrastructure SANs that don’t need the performance of FC Entry SANs that may not even need GigE and TOEs Hype overshadows reality: GigE NICs with iSCSI and TOEs cost ~ same as FC HBAs FC ports & GigE ports on server motherboards  Makes port cost differences higher for GigE w/TOEs Very low cost simplified FC switches  Have erased much of the infrastructure HW differences

20. Hosted by Sample iSCSI Vendors Switches Cisco Extreme Foundry Enterasys Nortel Lucent 3Com Gateways Cisco McDATA Silicon Adaptec Alacritech Intel Siliquent QLogic Storage NICs QLogic Intel Alacrite ch Adaptec Emulex

21. Hosted by By show of hands, is latency (delay) important to iSCSI block storage?

22. Hosted by Is Latency Important to IP Block Storage? The Answer is: Yes, for the most part It also depends on application

23. Hosted by By show of hands, who believes that TOEs & iSOEs are an iSCSI block storage requirement?

24. Hosted by Are TOEs & iSOEs an iSCSI block storage requirement? The Answer is:Not necessarily

25. Hosted by Spectrum of iSCSI Adapter Solutions Host Adapter Adapter Driver TCP/IP iSCSI SCSI Port to OS Software iSCSI “NIC + Driver” Media Interface Ethernet Media Interface Ethernet Fast Path TCP/IP Software iSCSI With Partial TCP Off-load TCP/IP iSCSI SCSI Port to OS Media Interface Ethernet TCP/IP iSCSI Firmware TCP and iSCSI Off-load SCSI Port to OS = SW or FW = Hardware Media Interface Ethernet TCP/IP iSCSI Hardware TCP and Firmware iSCSI Off-load SCSI Port to OS

26. Hosted by iSCSI: No TOE Definition Std Ethernet NIC  TCP/IP & iSCSI  Host-based in drivers Who Microsoft & Cisco Advantages Lowest cost NICs available today Easy integration with OS Disadvantages Lowest performance High CPU load High interrupts  Once/packet  Many/ TCP segment Adapter Driver TCP/IP iSCSI SCSI Port to OS Software iSCSI “NIC + Driver” Media Interface Ethernet = SW or FW = Hardware Host Adapter

27. Hosted by iSCSI: Little TOE Definition NIC w/limited TOE  Packets in order & no frags  Out of order etc. go to OS Who Alacritech Advantages Relatively low cost Small layout (low profile card) Good throughput w/pristine Ethernet Disadvantages Out-of-order & frags < performance Interrupts Once/TCP segment  Many/IO OS interface challenges Media Interface Ethernet Fast Path TCP/IP Software iSCSI W/Partial TCP Off-load TCP/IP iSCSI SCSI Port to OS Host Adapter = SW or FW = Hardware

28. Hosted by iSCSI: Firmware TOE + iSOE Definition TCP/IP & iSCSI firmware  On-board processors Who Adaptec, Intel, Emulex, QLogic Advantages Flexibility to change code Low CPU load Low interrupt load: < 1/IO Disadvantages No 10Gb scaling Performance Power, size Media Interface Ethernet TCP/IP iSCSI Firmware TCP and iSCSI Off-load SCSI Port to OS Host Adapter = SW or FW = Hardware

29. Hosted by iSCSI: Hardware TOE + iSOE Definition Hardware ASIC  TCP/IP bulk data path  iSCSI digest (CRC) iSCSI in processors Who QLogic, iReady Advantages Flexibility to change iSCSI code Low CPU load Low interrupt load < 1/IO Performance, scaling to 10G Disadvantages Complex chip Lack flexibility to change TCP code Media Interface Ethernet TCP/IP iSCSI Hardware TCP & Firmware iSCSI Off-load SCSI Port to OS = SW or FW = Hardware Host Adapter

30. Hosted by A TOEs Impact on iSCSI

31. Hosted by By show of hands, who believes that iSCSI allows block storage to go unlimited distance?

32. Hosted by Will iSCSI allow block storage to go unlimited distance? The Answer is:Yes & No Latency is the limiting factor Application dependent  Transactions cannot exceed 1ms one way (100 miles)  Asynch replication is not distance dependent “The speed of light, is not just a limit, it’s a law.”

33. Hosted by Key Block IP Storage Issues Distance Latency WAN bandwidth utilization of IP Security Encryption Access Performance Must be = to, or > than current expectations

34. Hosted by Storage Replication over WANs Issues Good Citizen on Shared TCP/IP WANs Filling the pipe > 50% End-to-end throughput  Compression  TCP latency

35. Hosted by Changing Paradigm for Asynch Storage Replication Native Storage GigE interfaces emerging EMC Symm5 and DMX are available today EMC CLARiiON in development Hitachi developing GigE for Lightning and Thunder Software Replication Apps over native IP Leverages IP WAN already in place Eliminates SAN gateway requirement (FCIP or iFCP) Significant < cost Mirror/Replication apps

36. Hosted by High Speed TCP/IP Data Transport Challenges Optimized for Small payloads & relatively short distances Employs inefficient Error recovery & session management techniques Delivers poor bandwidth utilization For most high performance applications Usually < 30% efficiency at extended distances Even less as distance and bit errors increase

37. Hosted by Cost of Inefficiency Higher Bandwidth Cost Despite < costs, high speed (DS3, OC3, etc) circuits = expensive  DSC survey of 200 end-users BW = 50-70 % of storage replication costs Operational Inefficiencies Can’t complete within time window delaying production ops Explosion in data exacerbates the problem Current = specialized equipment & separate networks  Can’t fully leveraging IP infrastructure = > costs

38. Hosted by Native GigE Replication: SRDF Adaptive Copy & SNAP/Asynch Performance degrades starting at ~ 300 miles  At 500 miles performance degradation is noticeable & significant Ethernet LAN/WAN Switches TCP/IP WAN EMC DMX SAN

39. Hosted by NetEx HyperIP Native GigE Replication: SRDF + RFC 3135 Adaptive Copy & SNAP/Asynch RFC 3135 = TCP/IP Performance Enhancing Proxy Up to satelite distances (46K miles roundtrip) 90% + bandwidth utilization (T1/E1, DS3, OC3, OC12) Plus 2 to 4 to 1 compression Who NetEx (HyperIP), Expand, NetCera, Digital Fountain Ethernet LAN/WAN Switches TCP/IP WAN EMC DMX SAN

40. Hosted by EMC SRDF Replication over WANs Replication Methodologies Illustrated Ethernet LAN/WAN Switches Cisco MDS 9216 CNT UESR 3000 Brocade 3xxx TCP/IP WAN McDATA/Nishan 3300 EMC DMX NetEx HyperIP = GigE = FC

41. Hosted by Considerations for Designing IP Storage Networks Separate LAN fabric Minimally, separate VLAN Layer 2 switching Best latency for Ethernet switching Nothing less than GigE Understand LUNs Mapping and Masking

42. Hosted by Questions?