Globus FTP Evaluation test Catania – 10/04/2001Antonio Forte – INFN Torino

Globus FTP GlobusFTP is the Globus implementation of the GridFTP Protocol. The GridFTP protocol is a superset of a GSI-enabled FTP protocol. Catania – 10/04/2001Antonio Forte – INFN Torino

Globus FTP Features A high throughput, reliable, secure and robust data transfer mechanism GSSAPI security (PKI and Kerberos) support Automatic negotiation of TPC buffer/window sizes Parallel data transfer Third-party control of data transfer Partial file transfer Reliable data transfer Replica catalogs mechanisms Catania – 10/04/2001Antonio Forte – INFN Torino

Globus FTP Implementation Globus Data Grid Alpha Release 2: a particular version of Globus containing a set of production libraries and tools that allow the use of GridFTP protocol. GSI WUFTP v0.4b5: a modified version of the wu-ftpd server, enhanced with GridFTP's security plus the GridFTP partial file transfer and parallel transfer capabilities. GSI NCFTP v0.3: a modified version of the ncftp client tools, enhanced with GridFTP's GSI-based security. Catania – 10/04/2001Antonio Forte – INFN Torino

Test INFN sites and resources INFN CNAFPentium II 400MHzWAN link 12Mbps INFN PADOVAPentium III 450MHzWAN link 16Mbps INFN NAPOLIPentium 200MHzWAN link 8Mbps INFN TORINOPentium Pro 200MHzWAN link 12Mbps Catania – 10/04/2001Antonio Forte – INFN Torino

Test layout Catania – 10/04/2001Antonio Forte – INFN Torino

Tests BASELINE MEASUREMENT ROUND TRIP TIME (RTT) Measurement of RTT with 500 icmp packets THROUGHPUT WITH STANDARD FTP (GET and PUT) Variable file size (1, 10, 50, 100, 500 MB) TESTS WITH GSIFTP PROTOCOL (using globus-url-copy command) COMPARISON BETWEEN STANDARD GSI FTP AND PARALLEL GSI FTP (GET and PUT) streams (1, 2, 4, 8, 16, 32, 64), fixed file size (100MB) streams (1, 2, 4, 8, 16, 32, 64), fixed file size (500MB) fixed streams (8), variable file size (1, 10, 50, 100, 500 MB) Catania – 10/04/2001Antonio Forte – INFN Torino

Tests EVALUATION OF GSIFTP PERFORMANCES VARING TCP BUFFER SIZE (GET and PUT) (Parameter: -tcp-bs buffer-size) fixed streams (1), variable TCP buffer size (4, 8, 16, 32, 64, 128, 256 KB) fixed streams (8), variable TCP buffer size (4, 8, 16, 32, 64, 128, 256 KB) EVALUATION OF GSIFTP PERFORMANCES VARING BLOCK SIZE (GET and PUT) (Parameter: -bs block size) fixed streams (1), variable block size (4, 8, 16, 32, 64, 128, 256 KB) fixed streams (8), variable block size (4, 8, 16, 32, 64, 128, 256 KB) Catania – 10/04/2001Antonio Forte – INFN Torino

Results asymmetrical throughput in different directions throughput increases for up to 16 streams and then keeps constant or, alternatively, decreases COMPARISON BETWEEN STANDARD GSI FTP AND PARALLEL GSI FTP (GET and PUT) Catania – 10/04/2001Antonio Forte – INFN Torino

Results WAN Link statistics the asymmetrical throughput depends on the different data path INFN Padova – Link 16 Mbps Saturation with 500 MB file size INFN Torino – Link 12 Mbps Catania – 10/04/2001Antonio Forte – INFN Torino

Results asymmetrical throughput in different directions throughput increases for up to 8 streams, then decreases COMPARISON BETWEEN STANDARD GSI FTP AND PARALLEL GSI FTP (GET and PUT) Catania – 10/04/2001Antonio Forte – INFN Torino

Results no significative throughput for 1 MB file size throughput slighty decreases for transfer of very large files (500 MB) COMPARISON BETWEEN STANDARD GSI FTP AND PARALLEL GSI FTP (GET and PUT) Catania – 10/04/2001Antonio Forte – INFN Torino

Results Optimal TCP buffer size is 32 KB EVALUATION OF GSIFTP PERFORMANCES VARING TCP BUFFER SIZE (GET and PUT) Catania – 10/04/2001Antonio Forte – INFN Torino

Results Optimal TCP buffer size is 32 KB Similar results in symmetrical tests (TO-NA, NA-TO) EVALUATION OF GSIFTP PERFORMANCES VARING TCP BUFFER SIZE (GET and PUT) Catania – 10/04/2001Antonio Forte – INFN Torino

Results No influence on the performance with 8 parallel streams EVALUATION OF GSIFTP PERFORMANCES VARING TCP BUFFER SIZE (GET and PUT) Catania – 10/04/2001Antonio Forte – INFN Torino

Results No influence on the performance (no CPU bottleneck) EVALUATION OF GSIFTP PERFORMANCES VARING BLOCK SIZE (GET and PUT) Catania – 10/04/2001Antonio Forte – INFN Torino

Results EVALUATION OF GSIFTP PERFORMANCES VARING BLOCK SIZE (GET and PUT) No influence … Catania – 10/04/2001Antonio Forte – INFN Torino

Advantages Catania – 10/04/2001Antonio Forte – INFN Torino No further authentication required (only grid-proxy-init needed) Throughput increases with parallel streams Third-party control of data transfer is possible Partial and realible transfer works fine with GSI FTP client (GSINCFTP)

Troubles No stable release (very frequently updated, we used the version downloaded on the 9 th Jan 2001) System crash with globus-url-copy, 64 streams and 500 MB of data Network interface sometimes in down state after parallel transfers of large files (500 MB) which frequently freeze; problems encountered with 16 streams between CNAF and Padova, with 32 or 64 streams between Torino and Napoli. These problems probably depend on bugs in the Linux kernel (we used ) Production WAN links negatively influenced the tests: dedicated links need!!! Catania – 10/04/2001Antonio Forte – INFN Torino

Latest release Globus Data Grid Alpha Release 3 released on the 26 th March 2001 (annunced by Lee Liming) not tested yet!!! Available at the following URL: alpha3.html Catania – 10/04/2001Antonio Forte – INFN Torino

Available at the WorkPackage 5 (Network) - INFN GRID Project homepage: More info Catania – 10/04/2001Antonio Forte – INFN Torino