DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories DART Workshop on Infrastructure Chief Investigator: Dr. Asad Khan Lead Investigator: Abdul Malik Khan 12 th April 2006 Melbourne

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories SI-8 Work Package Description Secure data replication via the network to DART Partners Introduction to FAST TCP FAST TCP Implementation Data –Storage –Encryptions –Transfer Mechanism Deliverables Links

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories At MU, JCU and UQ Data is collected from monitors, sensors, Synchrotron, & Barrier Reef Sensor Networks and stored at the Primary Datastore. Then the processed Data is securely transferred to Secondary Datastore Secure Data Replication via the Network to DART Partners Monash University James Cook University University of Queensland QPSF Grid Primary Datastore Secondary Datastore Sensor Network AARNet_GrangeNet

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories For efficient and high-speed data transfer the normal TCP needs modification and is replaced with FAST TCP. Fast Active queue management Scalable Transmission Control Protocol, a new algorithm for huge data transfer Monash University James Cook University University of Queensland QPSF Grid Primary Datastore Secondary Datastore Sensor Network AARNet_GrangeNet

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories  FAST TCP uses existing Internet.  FAST TCP is implemented as a software at the sending repository that parses the data into network- compatible packets that avoid typical network congestion as they reach the destination repository Monash University James Cook University University of Queensland QPSF Grid Primary Datastore Secondary Datastore Sensor Network AARNet_GrangeNet

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories Equilibrium Properties:- Uses both delay and loss. Define an equilibrium neighborhood around the target queuing delay (measure avg. RTT and minimum RTT). Rapid convergence when equilibrium changes (Monitor the per-ack queuing delay to avoid overshoot) Achieves high utilization. Stability Properties Stable for arbitrary delay, capacity, routing, and load Pacing is used to avoid queuing delay caused by busty traffic by incrementing congestion window Monash University James Cook University University of Queensland QPSF Grid Primary Datastore Secondary Datastore Sensor Network AARNet_GrangeNet

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories Monash University James Cook University University of Queensland QPSF Grid Primary Datastore Secondary Datastore Sensor Network AARNet_GrangeNet TCP Protocol Processing Estimation Loss ControlBurstiness Control RTT timescale Loss recovery RTT timescale Window Control Each component is: Designed independently & upgraded asynchronously Current TCP performs badly as networks continue to scale up in capacity and size. FAST tries to achieve high performance and stability. Each FAST TCP connection has an explicit equilibrium state, which enables it to achieve both stability and high performance. When FAST TCP is far away from this equilibrium, it rapidly converges toward the equilibrium, as shown in the figure. As FAST TCP approaches the equilibrium, it slows down to avoid overshoot, hence maintaining stability and performance. equilibrium fast convergence The data control component determines which packets to Transmit. The window control determines how many packets to Transmit, and The Burstiness control determines when to transmit these packets. These decisions are made based on information provided by the estimation component.

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories FAST TCP Implementation:  Linux integration (kernel compilation)  FASTHeader &  FASTModule Monash University James Cook University University of Queensland QPSF Grid Primary Datastore Secondary Datastore Sensor Network AARNet_GrangeNet

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories Monash University James Cook University University of Queensland QPSF Grid Primary Datastore Secondary Datastore Sensor Network AARNet_GrangeNet -Encryption to protect the data at rest -Encrypt data that moves. 3DES and newer algorithms like AES, RC6 and Blowfish. -Encryption of the channel (FTP via SFTP or HTTP via Secure Sockets Layer [SSL]). More appropriate than encrypting the source data files if the endpoints are secure. -Tool: GRID-FTP, Fast-Track FTP, Data-Encryption implementation. Storage systems: -Focus on high-performance access to data. -Utilize parallel data transfer streams and/or striping across multiple servers to improve performance. -Focus on high-volume/ large dataset usage. -Replication and local caching to divide and balance server load. File Transfer Mechanism: -Focus on common data transfer protocol. -Interoperability between storage systems. -supports PKI or Kerberos authentication. -Automatic negotiation of TCP buffer/window sizes. -Parallel data transfer. -Support for reliable data transfer.

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories Deliverables Setup of high speed link between the high- performance data centre’s Secure robust connectivity with availability, security, response time and throughput. Data transfer test of different data sets with varying data sets patterns. Simulated Network models for performance evaluation. Network performance models with what-if scenario’s. Documentation

DART SI-8: Pilot long-distance high speed and secure data transfer between the Repositories Links multi.html