1. Slide 1 DESIGN, IMPLEMENTATION, AND PERFORMANCE ANALYSIS OF THE ISCSI PROTOCOL FOR SCSI OVER TCP/IP By Anshul Chadda (Trebia Networks)-Speaker Ashish Palekar (Trebia Networks) Narendran Ganapathy (Sun Microsystems) R.D.Russell (University of New Hampshire)

2. Slide 2 Overview Background –SCSI Concepts –SAN –iSCSI –SCSI Subsystem in Linux iSCSI Initiator Implementation Resources Used Basic Performance Analysis Conclusions and Future Work Questions

3. Slide 3 What is SCSI? Acronym for Small Computer System Interface SCSI device identified by ID Logical Units present within the SCSI device Logical Unit Numbers: for logical addressing TARGET Hard disk INITIATOR (Host) LUN 0LUN 1LUN 2LUN 3 ID 7ID 0 SCSI bus

4. Slide 4 SCSI Terms Limitations Length of the SCSI bus (25 m) Number of SCSI devices supported Single host point-to-point INITIATORTARGET COMMAND STATUS MESSAGE IN MESSAGE OUT DATA IN DATA OUT

5. Slide 5 Storage Area Network (SAN) Host accesses the remote storage as a device Unit of Access: BLOCK Security, Consistency, Integrity Issues at BLOCK level SAN Application Client 1 Block Access Protocol Application Client 2 Disk

6. Slide 6 New SAN Approaches TCP/IP/ Ethernet Network Application Client 1 Block Access Protocol Application Client 2 Disk Ideal Solution Our Approach Application Client 1 Block Access Protocol Application Client 2 Disk TCP/IP/ Ethernet Network Target Emulator

7. Slide 7 iSCSI Protocol (an IETF standard) Requires Reliable Transport Layer (TCP) Session established between Initiator and Target Multiple connections within a session Header size –Large (48-byte) Basic header segment –Additional Header Segment can be present Sequence Numbers Header and Data Digests optional Security and Authentication optional Parameter Negotiation Target Discovery Mechanism

8. Slide 8 User space Kernel space Target Emulator Initiator User Application File System SCSI SCSI HBA Driver Disk User Application File System SCSI iSCSI Subsystem Disk Transport Network Network Stack SCSI iSCSI Subsystem SCSI HBA Driver Standard SCSI SCSI Over Network

9. Slide 9 SCSI Subsystem in Linux SCSI Upper level SCSI Mid level SCSI Host Bus Adapter (Low level driver) Target (Disk) Target (Disk) Target (Disk) Transport Network SCSI Host Bus Adapter (Low level driver) Target (Disk) Target (Disk) Target (Disk) Transport Network SCSI CommandSCSI Status, Response SCSI Command SCSI Status, Response

10. Slide 10 iSCSI Initiator Implementation

11. Slide 11 Resources Used For Testing Initiator –2.4 GHz CPU, 1GB Memory –Fast/Gigabit Ethernet Cards –Linux Kernel (2.4.19) –UNH iSCSI Initiator kernel modules Target –2.4 GHz CPU, 1GB Memory –Fast/Gigabit Ethernet Cards –Linux Kernel (2.4.19) –UNH iSCSI Target kernel modules

12. Slide 12 Performance Metrics & Variables Metrics: Bandwidth, Latency and % CPU utilization Variables –Initiator Scatter-Gather List Size –Ethernet Link Speed –Target Sector Size –Header & Data Digests –TCP Nagle Algorithm

13. Slide 13 Baseline Performance Values For the following Performance Variable values: –Initiator Scatter-Gather List Size=64 –Ethernet Link Speed=1000 Mbps –Target Sector Size=512 B –Header & Data Digests=OFF –TCP Nagle Algorithm=OFF READ: –Bandwidth=50.6761 MB/s, –%CPU util. On target=18.65% –%CPU util. On initiator=27% WRITE: –Bandwidth=53.3937 MB/s –%CPU util. On target=14.3% –%CPU util. On initiator=26.9%

14. Slide 14 Effect of Initiator Scatter-Gather list size on Bandwidth

15. Slide 15 Effect of Ethernet Link Speed on Bandwidth

16. Slide 16 Effect of Target Sector Size on Bandwidth

17. Slide 17 Effect of Header & Data Digests on Bandwidth

18. Slide 18 Effect of TCP Nagle Algorithm on Bandwidth

19. Slide 19 Conclusions To Increase Bandwidth –Increase Initiator Scatter-Gather List Size –Increase Ethernet Link Speed –Increase Target Sector Size –Turn OFF Header & Data Digests –TCP Nagle Algorithm has negligible effect

20. Slide 20 Future Work Run multiple simultaneous applications on the initiator system to utilize the CPU and bandwidth Do Latency measurements too Detailed Performance Analysis for each network stack involved Analyze the effect of iSCSI protocol parameters on performance. Test hardware implementation for TCP/IP and iSCSI Compare with Fiber Channel technology for bandwidth, latency and CPU utilization

21. Slide 21 Acknowledgements SUN/EMC for funding the project Inter-Operability Lab. And Trebia Networks for providing system resources

22. Slide 22 Questions???