1. A Data Engine for Grid Science Gateways Enabling Easy Transfers and Data Sharing Dr. Marco Fargetta (1), Mr. Riccardo Rotondo (2,*), Prof. Roberto Barbera (3,4) (1) Consorzio COMETA, Catania, Italy (2) Consortium GARR, Rome, Italy (3) Department of Physics and Astronomy of the University of Catania, Italy (4) INFN – National Institute of Nuclear Physics, Division of Catania, Italy (*) riccardo.rotondo@garr.it.

2. Outline Science Gateway Motivations Grid Data Management on SGs Data on Grid Services (DOGS) Technologies and implementations Conclusion & Outlook EGICF 2012, Munich2

3. Science Gateway Reference Model EGICF 2012, Munich3 Science Gateway Science Gateway Appl 1 Appl 2 Appl N Grid Services Users from different organisations having different roles and privileges Other Middleware

4. Science Gateway paradigm Efforts to grant easy yet secure access to grid services brought to the birth of Science Gateways Virtual Research Communities access remote resources in a collaboration environment that hides the underlying complexity SGs help many users to better use the enormous grid computational power Is large grid data storage accessible as well in such an easy way? EGICF 2012, Munich4

5. Motivations 5 Um… isn’t your computer on fire ? It’s ok, my files are stored in a safer place. Image source: 5

6. Grid Data Management Challenges Make interfaces simple for non expert users o CLI-based Grid storage interface is not straightforward Grid transactions require user certificates Complexity of current protocols to manage grid storage elements o Very little or no support for access through modern browsers or others web-based applications EGICF 2012, Munich6

7. Requirements Grid Storage complexity hidden to end users o Users move files from/to a portal and see it as simple external storage accessible from a web interface and do not care about grid (or any other) technologies behind File management smoothly integrated with all the services provided in the SG Underlining architecture exposes a file-system- like view (i.e., a Virtual File System or VFS) through which users can perform the following actions: o Create, move, delete files/directories with the desired structure o Share files with other users o Set the number of backup copies desired EGICF 2012, Munich7

8. Upload workflow EGICF 2012, Munich8 1. Sign in eTokenServer User Tracking DB DOGS DB 5. File Upload 3. Proxy request 4. Proxy transfer 7. Update DB 6. Upload on Grid 7. Tracking 2. Upload request

9. Data On Grid Services: DOGS A file browser shows Grid files in a tree File system exposed by the SG is virtual Easy transfer from/to Grid (by SG) is done in a few clicks Users do not need to care about how and where their files are really located EGICF 2012, Munich9

10. 10 Back-end technical details JSAGA API used to transfer data from/to storage elements Hibernate to manage the VFS collecting information on files stored on Grid; any changes/actions in the user view affect the VFS MySQL as underlying RDBMS An additional component has been developed in order to keep track of each transaction in the users tracking DB (to be compliant with the EGI Portal and User Traceability Policies)

11. Front-end technical details A portlet has been created to be deployed in a Liferay-based portal to which access is provided only to federated users with given roles and privileges o http://www.liferay.com http://www.liferay.com The portlet view component includes elFinder, a web-based file manager developed in Javascript using jQuery UI for a dynamic and user friendly interface o http://elrte.org/elfinder http://elrte.org/elfinder EGICF 2012, Munich11

12. The Data Engine in action EGICF 2012, Munich12

13. The Data Engine in action EGICF 2012, Munich13 «Share» to be added soon

14. The Data Engine in action EGICF 2012, Munich14

15. The Data Engine in action EGICF 2012, Munich15

16. Conclusions File transfer enabled with 100% browser friendly protocols Using JSAGA makes the Data Engine middleware independent User friendly interface All common file actions provided ( file sharing among users available soon) EGICF 2012, Munich16

17. Future Works Allow direct transfer to Grid storage element when the middleware will fully support web protocols (HTTP/HTTPS): o Avoid bottlenecks at the level of the Science Gateway; o Overcome file size limit (4 GB in Liferay 6.0.6) Provide a DataEngine remote interface for external applications. Support for mobile appliances (both iOS and Android based) EGICF 2012, Munich17

18. Thank you for your kind attention EGICF 2012, Munich18