Widening the number of e-Infrastructure users with Science Gateways and Identity Federations Giuseppe Andronico INFN - Italy Workshop on Science Applications And Infrastructure In Clouds And Grids– Oxford,15-16 March 2012

Outline  Introduction and driving considerations  The Science Gateway paradigm:  Architecture  Authentication and Authorisation Schema  Access workflow  Grid transaction model  The Authentication process  Use cases and statistics  The forthcoming Cloud Engine  Summary and conclusions 2

Path to technology uptake The Rogers “bell-shape” curve - Rogers, E. M. (1962), “Diffusion of Innovations”, Glencoe: Free Press. 3

IT acceptance model – the Web Davis, F. D. (1989), "Perceived usefulness, perceived ease of use, and user acceptance of information technology", MIS Quarterly 13(3): 319–340 Development of web browsers The World Wide Web 4

The evolution leap in web browsers 5 evolution leap 5

The eResearch2020 report ( Some barriers in the adoption of Grids:  Changes on Grids means changes on applications  Time required to adapt usual workflows  Lack of structure to support anonymous access  Download and installation of applications  Interface  Slow to get to compared to other resources  Difficult to use in the beginning  Time spent to get the application compiled and running 6

Using Grids is not straightforward  Users have to cope with complex security procedures, execution scripts, job description languages, command line based interfaces and lack of standards. This makes the learning curve very steep and keeps non IT-experts away.

Another consideration… VRCs # of users There is a huge number of non IT-experts out there who do not belong to any constituted Virtual Research Community. How can we attract them ? 8

I have a dream… Can we increase the number of potential grid users by a factor of 1,000 ? … or even by a factor of 25,000 and more ? 9

A new paradigm: the Science Gateway “A Science Gateway is a community-developed set of tools, applications, and data that is integrated via a portal or a suite of applications, usually in a graphical user interface, that is further customized to meet the needs of a specific community.” Teragrid 10

Davis, F. D. (1989), "Perceived usefulness, perceived ease of use, and user acceptance of information technology", MIS Quarterly 13(3): 319–340 Development of Science Gateway Requirement for sustainability IT acceptance model – the Grid 11

Primary requirement: building Science Gateways should be like playing with Sc. Gtwy E Sc. Gtwy DSc. Gtwy CSc. Gtwy B Sc. Gtwy A Standards Simplicity Easiness of use Re-usability 12

Our reference model Science Gateway Science Gateway App. 1 App. 2 App. N Embedded Applications Administrator Power User Basic User Users from different organisations having different roles and privileges Standard-based (SAGA) middleware-independent Grid Engine Standard-based (SAGA) middleware-independent Grid Engine 13

AuthN & AuthZ Schema AuthorisationAuthorisation Science Gateway GrIDP (“catch-all”) GrIDP (“catch-all”) IDPCT (“catch- all”) IDPCT (“catch- all”) IDP_y LDAP Register to a Service 2. Sign in Authentication Social Networks’ Bridge IdP

The Grid IDentity Pool (GrIDP) ( This is a “catch-all” Identity Federation

eduGAIN ( All the Science Gateways are registered as Service Providers of eduGAIN 16

17 Grid Engine Users Tracking DB Science GW Interface SAGA/JSAGA API Job Engine Data Engine Users Track & Monit. Science GW 1 Science GW 2 Science GW 3 Grid MWs Liferay Portlets eToken Server DONEBy end of April Catania Grid Engine By mid April DONE 17

Job Engine - Architecture WT Worker Threads for Job Submission WT Worker Threads for Status Checking USER TRACKING DB MONITORING MODULE GRID INFRASTRUCTURE(S) Job Queue WT Job Submission Job Check Status/ Get Output 18

19 Job Engine - Features  The Job Engine has been designed with the following features in mind: FeatureDescriptionStatus Middleware Independent Capacity to submit job to resources running different middleware DONE EasinessCreate code to run applications on the grid in a very short time DONE ScalabilityManage a huge number of parallel job submissions fully exploiting the HW of the machine where the Job Engine is installed DONE PerformanceHave a good response timeDONE Accounting & Auditing Register every grid operation performed by the usersDONE Fault ToleranceHide middleware failure to end usersALMOST DONE WorkflowProviding a way to easily create and run workflowsIN PROGRESS

20 Job Engine – Scalability 40,000 jobs submitted in parallel ! Time to submit 10,000 jobs (h) Job submission time (h)  Submission time scales linearly with number of jobs  >10,000 jobs a hour 20

Both sequential and MPI-enabled jobs successfully executed Tests with Globus planned 21 Job Engine – Middleware interoperability

Job Engine – Accounting & Auditing  A powerful accounting & auditing system is included in the Job Engine  It is fully compliant with EGI VO Portal Policy and EGI Grid Security Traceability and Logging Policy  The following values are stored in the DB for each job submitted:  User ID  Job Submission timestamp  Job Done timestamp  Application name  Job ID  Robot certificate ID  VO name  Execution site (name, latitude, longitude) 22

Catania Science Gateways in numbers Overall usage (arb. units) 23

Data Engine – Requirements  A file browser shows Grid files in a tree  File system exposed by the Science Gateway is virtual  Easy transfers from/to Grid (through the SG at the moment) are done in a few clicks  Users do not need to care about how and where their files are really located 24

Data Engine – Usage Workflow Sign in eTokenServer User Track. DB DOGS DB 5. File Upload 3. Proxy request 4. Proxy transfer 6. Update DB 7. Upload on Grid 7. Tracking 2. Upload request 25

DOGS: Data On Grid Services – Back-end implementation 26  JSAGA API used to transfer data from/to storage elements  Hibernate to manage the VFS collecting information on files stored on Grid; any changes/actions in the user view affect the VFS  MySQL as underlying RDBMS  An additional component has been developed in order to keep track of each transaction in the users tracking DB

DOGS: Data On Grid Services – Front-end implementation  A portlet has been created wit access provided only to federated users with given roles and privileges  The portlet view component includes elFinder, a web-based file manager developed in Javascript using jQuery UI for a dynamic and user friendly interface 

Data Engine in action (1/2) 28

Data Engine in action (2/2) «Share» to be added soon 29

Summary of standards adopted  The framework for Science Gateways developed at Catania is fully web-based and adopts official worldwide standards and protocols, through their most common implementations  These are:  The JSR 168 and JSR 286 standards (also known as "portlet 1.0" and "portlet 2.0" standards)JSR 168JSR 286  The OASIS Security Assertion Markup Language (SAML) standard and its Shibboleth and SimpleSAMLphp implementationsOASISSecurity Assertion Markup Language ShibbolethSimpleSAMLphp  The Lightweight Direct Access Protocol, and its OpenLDAP implementationOpenLDAP  The Cryptographic Token Interface Standard (PKCS#11) standard and its Cryptoki implementationCryptographic Token Interface Standard  The Open Grid Forum (OGF) Simple API for Grid Applications (SAGA) standard and its JSAGA implementationOpen Grid ForumSimple API for Grid ApplicationsJSAGA 30

INDICATE Review Roberto Barbera Lyon, 20/09/ Science Gateways in action: e-Culture Science INDICATE

Use the HTTPS interface of Storage Elements Important for large-size files Science Gateways in action: e-Culture Science INDICATE 32

Science Gateways in action: e-Culture Science INDICATE

Thanks to the collaboration with Science Gateways in action: e-Culture Science INDICATE

Science Gateways in action: EUMEDGRID

Science Gateways in action: GISELA 36

Science Gateways in action: DECIDE 37

The CHAIN Application Database ( Project-specific Science Gateways can be accessed from the CHAIN AppDB 38

Cloud Engine Users Tracking DB OCCI API Users Track & Monit. Cloud App 1 Cloud App 2 Cloud App N Cloud MW Cloud Gateway The forthcoming Cloud Engine AWS

Host Management Layer: Host Manager Performs physical resources monitoring and VEs allocation Cluster Management Layer: Cluster Manager Monitoring the overall state of the cluster, “coordinates” HMs External components: XMPP Server and Distributed Database XMPP advantages: host presence, open standard Central failure point does not exist: fault tolerance mechanism with multiple CM instances Virtual execution environment: CLEVER

Summary and conclusions  e-Infrastructures can be very beneficial platforms (especially for cultural heritage), provided they are really «easy to use»  Science Gateways with support for Identity Federations and Social Networks can revolutionize the way Grid infrastructures are used, hugely widening their potential user base, especially non-IT experts and the “citizen scientist”  The adoption of standards (JSR 286, SAGA, SAML, etc.) represents a concrete investment towards sustainability  By design, the components (the “portlets” – our “Lego bricks”) of our Science Gateways have maximum re-usability and, indeed, they have been already adopted in/by several projects (CHAIN, DECIDE, EarthServer, EUMEDGRID-Support, GISELA, INDICATE, etc.)  If you want to integrate your applications in our Science Gateways, or simply enable your websites with our authentication tools, please contact me at 41

Thank you 42