GridFTP: File Transfer Protocol in Grid Computing Networks Caitlin Minteer

Agenda Grid Computing Globus Toolkit Grid FTP Advantages of GridFTP Disadvantages of GridFTP Using GridFTP GridFTP Clients

Grid Computing and the Globus Toolkit Grid computing is the rising networking infrastructure that is designed to offer access to computational data and human resources spread over wide area environments The Globus Toolkit is a technology for the grid open source toolkit building computing grids

GridFTP GridFTP - File Transfer Protocol in Grid Computing Networks. high-performance secure reliable data transfer protocol optimized for high-bandwidth wide-area networks. based upon the Internet FTP protocol it implements extensions for high-performance operation it implements extensions for high-performance operation already specified in FTP specification

GridFTP Provides: A highly extensible server a scriptable command-line client a set of development libraries for custom solutions.

Advantages of GridFTP Security Parallel Streams Striping Partial File Transfer Reliable and Restartable data transfer Data Extensibility Protocol Extensibility

GridFTP: Security Authentication of users or services Protecting communications Determining authorization Managing user credentials and maintaining group membership information. The Globus GridFTP server and client use the Grid Security Infrastructure (GSI) protocol that allows a secure Public Key Infrastructure (PKI) interface, and adds the capability of delegated authority through certificates. determining authorization which is who is allowed to perform what actions

Parallel Streams GridFTP supports multiple TCP streams in parallel between a single source and destination. This feature can improve aggregate bandwidth in relation to that done by a single stream.

Stripping Stripping - having several network endpoints at the source, destination, or both participating in the transfer of the same file. Done by having a cluster with a parallel shared file system. Each node in the cluster reads a section of the file and sends it over the network. Striping and parallelism may be used together where one may have more than one TCP streams open between each of the servers participating in the transfer.

Partial File Transfer Partial file access: Regions of a file may be accessed by specifying an offset into the file and the length of the block desired GridFTP supports this capability by specifying the byte position in the file to begin the transfer. In many cases in the scientific community it is expedient to download only portions of a large file, instead of the entire file.

Third Party Control and Reliable/Restartable Data Transfer To enable reliability, the GridFTP server automatically sends restart markers (checkpoints) to the client. If the transfer has a fault, the client may restart the transfer and provide the markers received. The server will restart the transfer, picking up where it left off based on the markers. The Reliable File Transfer (RFT) service goes one step further by providing a service interface (job submission like interface) and writing the restart markers to a database so that it can survive a local fault. clients are also able to act as a third-party to initiate transfers between remote sites.

Data Extensibility The Data Storage Interface (DSI) module knows how to read and write to the local storage system and can optionally transform the data. It completely abstracts away the underlying storage.

Protocol Extensibility GlobusXIO – is the eXtensible Input/Output (XIO) framework in the Globus Toolkit. provides a simple abstraction layer to runtime loadable IO implementations. system uses a read, write, open, close abstraction that Globus GridFTP is able to leverge in order to be transport protocol agnostic. Therefore, protocols much more aggressive than TCP can be used. To meet more specific extensibility needs, we also provide easy-to-use development libraries. The eXtensible Input/Output (XIO) framework in the Globus Toolkit is the GlobusXIO

Disadvantages of GridFTP The client needs to remain active at all the times until the transfer finishes, which means that when the client state has been lost the rich set of recovery features of GridFTP can not be used. In the event of client state loss, transfer has to restart from scratch. GridFTP’s many features are tied to its protocol and implementation. Reimplementation and re-engineering would be required to provide these features to other file transfer services. several memory leaks, unclear error responses and bugs that have caused many issues in the use of GridFTP

Using GridFTP: Put, Get, & Third Party “Putting” – move a file from one system to a server ‘globus-url-copy -vb -tcp-bs 2097152 -p 4 file:///filename gsiftp://hostnameofserver/filename.’ “Getting” – move a file from the server to the local machine ‘globus-url-copy -vb -tcp-bs 2097152 -p 4 gsiftp://hostnameofserver/filename file:///filename.’ Third party transfers – move a file between two GridFTP servers globus-url-copy -vb -tcp-bs 2097152 -p 4 gsiftp://othermachinehostname/filename gsiftp://localhostname/filename. One of the most basic functions of GridFTP is to move a file from one file system to a server, also known as “putting.”

GridFTP Clients globus-url-copy – the provided scriptable, command line client Easy to use access multiple protocols that you can specify in a URL To use the globus-url-copy, a proxy certificate must be obtained. Then a temporary proxy must be generated  Globus does not provide an interactive client for GridFTP neither GUI nor text based. regular FTP clients will work with GridFTP but will not take advantage of all the features of GridFTP UberFTP is the first interactive, GridFTP-enabled ftp client. supports GSI authentication parallel data channels third party transfers.

Summary Grid Computing Globus Toolkit Grid FTP Advantages of GridFTP Security Parallel Streams Striping Partial File Transfer Reliable and Restartable data transfer Data Extensibility Protocol Extensibility Disadvantages of GridFTP Using GridFTP GridFTP Clients