1. A Sneak Peak of What’s New in Globus GridFTP John Bresnahan Michael Link Raj Kettimuthu (Presenting) Argonne National Laboratory and The University of Chicago

2. GridFTP l A secure, robust, fast, efficient, standards based, widely accepted data transfer protocol l We supply a reference implementation: u Server u Client tools (globus-url-copy) u Development Libraries l Independent implementations interoperate u Fermi Lab has a home grown server that work with ours l Lots of people have developed clients independent of the Globus Project

3. GridFTP l Two channel protocol like FTP l Control Channel u Communication link (TCP) over which commands and responses flow u Low bandwidth; encrypted and integrity protected by default l Data Channel u Communication link(s) over which the actual data of interest flows u High Bandwidth; authenticated by default; encryption and integrity protection optional

4. GridFTP DPISPI DPISPI CPI

5. Striping l GridFTP offers a powerful feature called striped transfers (cluster-to-cluster transfers)

6. Topics for discussion l Performance enhancement u GridFTP over UDT l Ease of Use enhancements u GridFTP over SSH u GridFTP Where there’s FTP l Resource Management in GridFTP l Future directions

7. GridFTP over UDT l UDT is an application-level data transport protocol that uses UDP to transfer data l Implement its own reliability and congestion control mechanisms l Achieves good performance on high-bandwidth, high- delay networks where TCP has significant limitations l GridFTP uses Globus XIO interface to invoke network I/O operations

8. GridFTP over UDT l XIO framework presents a standard open/close/read/write interface to many different protocol implementations u including TCP, UDP, HTTP -- and now UDT l The protocol implementations are called drivers. u A driver can be dynamically loaded and stacked by any Globus XIO application. l Created an XIO driver for UDT reference implementation l Enabled GridFTP to use it as an alternate transport protocol

9. GridFTP over UDT Argonne to NZ Throughput in Mbit/s Argonne to LA Throughput in Mbit/s Iperf – 1 stream19.774.5 Iperf – 8 streams40.3117.0 GridFTP mem TCP – 1 stream16.463.8 GridFTP mem TCP – 8 streams40.2112.6 GridFTP disk TCP – 1 stream16.359.6 GridFTP disk TCP – 8 streams37.4102.4 GridFTP mem UDT179.3396.6 GridFTP disk UDT178.6428.3 UDT mem201.6432.5 UDT disk162.5230.0

10. Alternate security mechanism l GridFTP traditionally uses GSI for establishing secure connections l In some situations, preferable to use SSH security mechanism l Leverages the fact that an SSH client can remotely execute programs by forming a secure connection with SSHD

11. GridFTP over SSH l sshd acts similar to inetd l control channel is routed over ssh u globus-url-copy popens ssh u ssh authenicates with sshd u ssh/sshd remotely starts the GridFTP server as user u stdin/out becomes the control channel

12. SSHFTP Interactions sshd CPI GridFTP Server 2811 Port 22 ROOT USER ssh Stdin/out

13. GridFTP Where there’s FTP (GWFTP) l GridFTP has been in existence for some time and has proven to be quite robust and useful l Only few GridFTP clients available l FTP has innumerable clients l GUI Clients? l Windows Clients?

14. GWFTP l GWFTP - created to leverage the FTP clients l A proxy between FTP clients and GridFTP servers l Not secure from client to proxy u Run on a trusted net (127.0.0.1) u Data channel routed or direct u If 3pt it is direct and secure u If 2 party must route through proxy, or be insecure

15. GWFTP (3pt) DPISPI DPISPI Your Client FTP 959 (not secure) GSI Credential GSI Delegated Credential gwtftp GSI Credential

16. GWFTP (2pt routed) DPI SPI Your Client FTP 959 (not secure) GSI Credential gwtftp GSI Credential DPI

17. GWFTP (2pt direct) DPI SPI Your Client FTP 959 (not secure) No Security gwtftp GSI Credential DPI

18. Resource management l Fork/Exec is safer service model u sandboxes leaks/segfaults/security/etc u If 1 session dies service exists l Transient state u We need permanent & shared state between sessions

19. GFork Server Host GFork Server GridFTP Plugin GridFTP Server Instance Fork GridFTP Server Instance GridFTP Server Instance State Sharing Link Client Inherited Links Control Channel Connections Client

20. Dynamic Backends l Dynamic list of available backends (DPIs) l Frontend (SPI) listens for registration u Backends register (and timeout) u Select backend(s) to use for a transfer l Backend failure is not system failure l Resources can be provisioned to suit load

21. Dynamic Backends Frontend Host GFork Server GridFTP Plugin Frontend Instance Fork Lookup available backend Registration Control Connection Backend Host GFork Server GridFTP Plugin Backend Instance Fork

22. Future directions l Resource Properties u GridFTP server expose state via resource properties l Server load l Connection limits u Act as a WS-MDS provider l Firewall traversal u Simultaneous open u Capability to make use of dynamic firewall port opening