Presentation is loading. Please wait.

Presentation is loading. Please wait.

Middleware Support for RDMA-based Data Transfer in Cloud Computing Yufei Ren, Tan Li, Dantong Yu, Shudong Jin, Thomas Robertazzi Department of Electrical.

Similar presentations


Presentation on theme: "Middleware Support for RDMA-based Data Transfer in Cloud Computing Yufei Ren, Tan Li, Dantong Yu, Shudong Jin, Thomas Robertazzi Department of Electrical."— Presentation transcript:

1 Middleware Support for RDMA-based Data Transfer in Cloud Computing Yufei Ren, Tan Li, Dantong Yu, Shudong Jin, Thomas Robertazzi Department of Electrical and Computer Engineering Stony Brook University

2 Outline n Introduction and Background n Middleware Design and RFTP application n Experimental Results n Conclusion

3 Outline n Introduction and Background l Overview l RDMA Semantics n Middleware Design and RFTP application n Experimental Results n Conclusion

4 Today’s Data-intensive Applications n Explosion of data, and massive data processing n Scalable storage systems n Ultra-high speed network for data transfer: 40/100Gbps networks n Reliable Transfer (error checking and recovery) at 40/100G speed, burden on processing power

5 ANI Ultra-high Speed Network

6 End-to-End 40/100G Networking 100G APPS FTP /100G NIC 40/100 Gbps Backbone 40/100 Gbps Backbone 100 G APPS FTP /100G NIC End-to-End Networking at 40/100 Gbits/s Our project and its role

7 Protocol Offload and Hardware Acceleration n TCP/IP Offload Engine (TOE) n Protocol Offload Engine (POE) n Remote Directory Memory Access (RDMA) l Kernel by pass l Zero-copy

8 Applications over different RDMA implementations

9 RDMA Semantics n Channel Semantic – SEND/RECV l Two-side operation l Both data source and data sink are involved. The sink pre- posts a list of buffers into receive queue. n Memory Semantic – RDMA WRITE/RDMA READ l One-side operation l Credit-based. The sink advertises its available registered memory to the source for RDMA_WRITE operation. n We use RDMA WRITE operation to deliver user payload(128KB ~ 4MB per block), while use SEND/RECV to exchange control messages( ~2KB).

10 Outline n Introduction and Backgroud n Middleware Design and RFTP application l Middleware Layer l Middleware Software Architecture l Asynchronous Communication Events design l RFTP Modules l RDMA extension to standard FTP protocol n Experimental Results n Conclusion

11 Middleware Layer InfiniBandRoCEiWARP IB Verbs libibverbs RDMA CM librdmacm Application Buffer Management Connection Management Event Dispatch/Join Task Scheduling Middleware OFED Hardware

12 Middleware – Multi-threaded Architecture ThreadsData Structure CQ QP-1QP-2QP-n Data Block List Receive Control Message List Send Control Message List Remote MR Info List application system Queue Pair List Memory Sender CE dispatcher CE slave-n... CE slave-2 CE slave-1 Logger Hardware HCA

13 Communication Events n Session ID negotiation l Each data transfer task will be assigned a unique session ID n Number of data connection negotiation l Establish several parallel connections n Memory region credit request and response l The source issues request of Memory regions’ information l The sink feedbacks several credit according to buffer status n Block completion notification l The source issues a notification to the sink which block’s data is ready

14 Parallel and Pipelined Data Transfer n Explore parallelism of RDMA operations l Multiple active data streams l Each stream uses a pipelined execution n Out-of-order blocks l Reorder l Deliver in-order blocks to application

15 RDMA-enabled FTP - RFTP RDMA Middleware FTP … Disk I/O Module InfiniBandiWARPRoCE Verbs Communication manager SSDMagnetic Disk Driver API Hardware Operating System Middleware Application Buffer Manage I/O Scheduling Connection Manage Event Dispatch Task Scheduling Direct I/O API

16 RDMA extension to standard FTP protocol

17 Outline n Introduction and Backgroud n Middleware Design and RFTP application n Experimental Results l Testbed Setup l LAN results l MAN results n Conclusion

18 Testbed Setup - LAN 10Gbps 40Gbps

19 Testbed Setup - MAN 40Gbps RoCE link RTT = 3.6ms

20 LAN – Bandwidth and CPU Usage Comparison

21

22 MAN – RFTP evaluation

23 Outline n Introduction and Background n Middleware Design and RFTP application n Experimental Results n Conclusion

24 Conclusion n Data-intensive application in cloud computing require efficient data transfer protocols to fully utilize the capacity of advanced network infrastructure n Designed and implemented a RDMA-based middleware layer n Developed a FTP application based on this middleware layer n Tested the performance of our design and implementation on both LAN and long-haul MAN links

25 Thank you


Download ppt "Middleware Support for RDMA-based Data Transfer in Cloud Computing Yufei Ren, Tan Li, Dantong Yu, Shudong Jin, Thomas Robertazzi Department of Electrical."

Similar presentations


Ads by Google