1. CGW 04, Stripped replication for the grid environment as a web service1 Stripped replication for the Grid environment as a web service Marek Ciglan, Ondrej Habala, Ladislav Hluchý Institute of informatics Slovak Academy of Sciences

2. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service2 Overview Replication in Grid environment Principles of stripped replication (SR) method Optimization of stripped replication Prototype Implementation as a Web Service Experimental Results Future Work

3. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service3 Replication in Grid environment Creation of multiple copies of single data source across Grid infrastructure Replication increases data availability RLS - Replica Location Service Grid monitoring services – network monitoring

4. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service4 Replication in Grid environment Storage Element 1 File 1 Storage Element 2 Storage Element 3

5. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service5 Replication in Grid environment Storage Element 1 File 1 Storage Element 2 Storage Element 3 File 1

6. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service6 Replication in Grid environment Storage Element 1 File 1 Storage Element 2 Storage Element 3 File 1

7. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service7 Replication in Grid environment Storage Element 1 File 1 Storage Element 2 Storage Element 3 File 1

8. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service8 Stripped Replication - Principles Transfer from multiple Grid sites, in parallel Transfer only a portion of file from each Storage Element (SE) Different file portions (stripes) are obtained from different SEs Parallel transfer increases replication speed If SR is not managed properly, process could be time consuming Optimization of SR management is required

9. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service9 Stripped Replication - Optimization Replicated data source Replica 1Replica 2Replica 3

10. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service10 Stripped Replication - Optimization Replicated data source Replica 1Replica 2Replica 3 Replica 1Replica 2Replica 3

11. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service11 Stripped Replication - Optimization Replica 1Replica 2Replica 3

12. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service12 Stripped Replication - Optimization Replica 1Replica 2Replica 3 Replica 1Replica 2Replica 3

13. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service13 Stripped Replication - Optimization Replica 1Replica 2Replica 3 Replica 1Replica 2Replica 3 Replica 1Replica 2Replica 3

14. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service14 SR Prototype Implementation Java programming language CoG 1.2 API (GridFTP interface) Integrated with EDG Replica Location Service EDG RLS API (RLS interface) File Chunks – basic data units for transfer Implemented as a Web Service ( motivation :OGSA, WSRF)

15. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service15 Service Workflow Stripped Replication Service LFN Get GUID

16. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service16 Service Workflow LFN Get GUID Replica Metadata Catalog Stripped Replication Service

17. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service17 Service Workflow LFN Get GUID Replica Metadata Catalog Stripped Replication Service Get PFNs

18. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service18 Service Workflow LFN Get GUID Replica Metadata Catalog Get PFNs Local Replica Catalog Stripped Replication Service

19. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service19 Service Workflow LFN Get GUID Replica Metadata Catalog Get PFNs Local Replica Catalog Stripped Replication Algorithm Stripped Replication Service

20. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service20 Service Workflow LFN Get GUID Replica Metadata Catalog Get PFNs Local Replica Catalog Stripped Replication Algorithm GridFTP Site 1 GridFTP Site N... Stripped Replication Service

21. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service21 Service Workflow LFN Get GUID Replica Metadata Catalog Get PFNs Local Replica Catalog Stripped Replication Algorithm GridFTP Site 1 GridFTP Site N... Register Replica Stripped Replication Service

22. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service22 Properties of Stripped Replication Parallel transfer from multiple sites increases replication process speed Proposed optimization does not use network monitoring services SR adapts to varying nature of network load SR optimally distributes network load

23. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service23 Experimental Results Motivation test case –File size 223.9Mb –Best replica transfer with standard replication tool (EDG rm) - 713 sec –Stripped replication (2 replicas) – 405 sec (43 %) –Stripped replication (3 replicas) – 209 sec (71 %) Average time saving –2 replicas – 37% time saving –3 replica – 55% time saving

24. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service24 Future Work Implementation refinement –Add logging functionality –Refine error states handling Evaluation of SR integration in Grid projects

25. Stripped Replication for Grids CGW 04, Stripped replication for the grid environment as a web service25 Thank you for your attention !