1. SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481 Science Gateways for improving the uptake of Grid and Cloud Computing Infrastructures Tamas Kiss, Stephen Winter, Gabor Terstyanszky, Thierry Delaitre University of Westminster Wibke Sudholt, CloudBroker GmbH Peter Kacsuk, Zoltan Farkas, SZTAKI Jens Krüger, Sandra Gesing, University of Tübingen

2. Using Distributed Computing Infrastructures is not so easy gsissh -p 2222 grid-compute.oesc.ox.ac.uk globus-job-submit grid-compute.oesc.ox.ac.uk/jobmanager- pbs -q RXXXXX /bin/hostname -f globus-job-status globus-job-get-output globus-job-clean ? ! or is it?

3. SCI-BUS is a 3-year EU FP7 project Started on 1 st October 2011 Build around 30 science gateways to various user communities in Europe 10 academic and 5 commercial partners from 10 European countries. SCIentific Gateway Based User Support

4. Main objectives of SCI-BUS Create a generic-purpose gateway technology Elaborate an application-specific gateway building technology and customisation methodology Establish production gateway services both for NGIs and various user communities Provide seamless access to major computing, data and networking DCIs and services including supercomputers, clusters, grids and clouds Create and maintain a Liferay portlet repository that enables the quick creation of user specific customised gateways Provide gateway development and application development support Develop business models to guarantee the sustainability of the gateway services and to enable the commercial exploitation

5. Value brought by the project SCI-BUS will serve large number of user communities by providing generic-purpose and customized scientific gateways for them. SCI-BUS will also create and provide access to many different portlet and application repositories to help user communities in sharing and running these applications via the SCI-BUS gateways

6. Project partners: International seismology community Helio-physics community Swiss systems biology community of the SystemsX.ch project German MoSGrid computational chemistry and bioinformatics community Biomedical researchers community of the Academic Medical Centre of the University of Amsterdam Astrophysics community PireGrid SMEs community Business process modelling community involving a wide range of areas such as finance, healthcare, government, production, robotics and emergency Blender rendering community Citizen web-2 community Public application developer community Subcontractors: 6 sub-contractors to be selected representing additional user communities Hugely competitive – over 3 times as much applications Additional user communities supported by the project The SCI-BUS user communities

7. The gateway services will enable unified access to all the major DCI infrastructures used in Europe: Local and remote clusters accessed by queuing systems (PBS, LSF, Condor, SGE) Local and remote supercomputers Local and volunteer Desktop Grids (BOINC, XtremWeb, OurGrid) Grids (ARC, gLite, Globus, UNICORE) Academic Public and Private Clouds (OpenNebula, Eucalyptus, etc.) Public Commercial Clouds (Amazon EC2, IBM Cloud, etc.) The SCI-BUS architecture

8. 8 The SCI-BUS Infrastructure

9. The baseline technology P-GRADE Portals around the globe –National Grid portals (UK, Ireland, Croatia, Turkey, Spain, Belgium, Malaysia, Kazakhstan, Switzerland, Australia, ClGrid, etc.) –HP-SEE/SEE-GRID-SCI, BalticGrid –US Open Science Grid, TeraGrid –IDGF (International Desktop Grid Federation) –SHIWA Simulation Platform –Economy-Grid, Swiss BioGrid, Bio és Biomed Virtual Organisations, MathGrid, etc. HP-SEE

10. P-GRADE Portal software family P-GRADE Portal 2.4 NGS P-GRADE portal P-GRADE Portal 2.5 P-GRADE Portal 2.8 P-GRADE Portal 2.9.1 WS-PGRADE Portal 3.1 WS-PGRADE Portal 3.2 GEMLCA Grid Legacy Code Arch. GEMLCA, storage concepts Basic concepts 2008 2009 2010 WS-PGRADE Portal 3.3 P-GRADE Portal 2.10 Open source since 2008 Open source since 2011 2011 WS-PGRADE Portal 3.4

11. Motivations of creating WS- PGRADE/gUSE To overcome (most of) the limitations of P-GRADE portal: To provide better modularity to replace any service To improve scalability to millions of jobs To enable advanced dataflow patterns To interface with wider range of resources To separate Application Developer view from Application User view WS-PGRADE (Web Services Parallel Grid Runtime and Developer Environment) and gUSE (Grid User Support Environment) architecture

12. WS P-GRADE in a nutshell General purpose, workflow-oriented portal. Supports the development and execution of workflow-based applications Based on Liferay Services supported by the portal: New functionalities – Web services – DB connectors – Embedded workflows – Job level PS – Conditional jobs – Recursive graph – Multi-generator – Multi-collector – CROSS product PS – DOT product PS Basic ServicesEGEE grids (LCG2,GLite) Globus grids (GT2,GT4) Desktop grids clusters Job execution File storage Certificate management Information system Brokering Job monitoring Workflow & job visualization

13. Inside of the System gUSE tires WF Graph editor WEB-UI (HTML) Liferay WS-PGRADE portal Liferay WS-PGRADE portal Information System WF Storage File Storage Application Repository WF Interpreter Backend local submitter GT4 submitter Glite submitter LCG submitter GT2 submitter GEMLCA submitter WS (Axis) submitter Glite Grid GMLCA Grid GT2 Grid GT4 Grid LCG Grid WS DCI-Bridge

14. WS P-GRADE workflow concept Concrete Workflow Algorithms, Resource references, Inputs Graph Jobs, Edges, Ports Template Constraints, Comments, Form Generators Workflow Instance Running state, Outputs Repository Item Application OR Project OR, Workflow part (G,T,CW) Legend: a b a must reference b a b a may reference b

15. ASM Application Specific Module A new module called Application Specific Module has been developed for gUSE that ease the creation of Application Specified Portlets – The module provides programming API to control applications developed using gUSE – The module provides programming API to access inner gUSE services Any kind of Graphical interface can be developed on top of this module – GUI written in JSP, JSTL (Java Servlet Tag Library ) and Java – GUI can be customized for the grid application – GUI can be customized for end users

16. ASM - Application Specific Module DCI systems Grid Application Developer creates own Grid Application via ws-PGrade and export it to the local repository gUSE Repository ws-PGrade Grid Application Submitter Workflow Interpreter Workflow Storage Storage Information User space Grid Application Then the Grid Portal Developer creates a new web-application (mainly an web-interface) which fits to the user community's needs, and connects it to the gUSE using ASM API Developed web- interface ASM Finally the end-users create own application from the exported one, parameterize and execute it using the developed portlet > Grid Application

17. CloudBroker Platform Easy, scalable, secure, integrable and pay-per-use access to scientific and technical applications in the cloud High performance computing application store and marketplace with accounting and billing Deployment and execution of applications in the cloud with the same parameters and files as for local execution Accessible through the internet via a web browser UI Integration possibilities into third party tools via web service-based APIs Public version available under https://platform.cloudbroker.comhttps://platform.cloudbroker.com Using infrastructure as a service (IaaS) from cloud providers Offering platform as a service (PaaS) for software vendors Providing software as a service (SaaS) to end users CloudBroker GmbH

18. Platform Architecture CloudBroker GmbH CloudBroker Platform Amazon Cloud IBM Cloud … Cloud Chemistry Appli- cations Biology Appli- cations Medicine Appli- cations Web Browser UI Engineering Appli- cations Web Service API Generic Workbenches CloudBroker Integration Domain-Specific Gateways R&D End Users and Software Vendors CLI … Appli- cations

19. CloudBroker Business Model Free access to the platform after registration Each organization manages its own users Prepayment or monthly billing Platform usage costs = resource usage costs + software usage costs + CloudBroker usage surcharges CloudBroker GmbH Cloud Broker End Users Software Vendors Cloud Providers ResourcesApplications Usage$ $$

20. Clouds and Applications Currently supported cloud infrastructures: – Amazon Web Services – IBM SmartCloud Enterprise Adapters to further clouds planned Open source-based private cloud tools will be interfaced within SCI-BUS Current applications: – Chemistry (GAMESS, Gromacs) – Biology (BLAST, X! Tandem, Rosetta) – Medicine (AutoDock) – Engineering (OpenFOAM) Further applications in preparation Own applications can be easily added by users CloudBroker GmbH

21. CloudBroker Platform Integration in Liferay/WS-PGRADE/gUSE Gamess GromacsAutodockOpen Foam Blast Rosetta Xtandem... API gUSE WS-PGRADE Job Configuration Portlet CloudBroker UI Portlet UI DCI-BRIDGE SaaS (Softwares as a Service) CloudBroker plugin Expose the CloudBroker UI through WS-PGRADE interface by proxy. User Mapping Portlet CB Java API Integrate the SaaS capabilities of CB into the gUSE/WS-PGRADE system. Enable WS-PGRADE users to run softwares (SaaS) exposed by the CB platform.

22. MosGrid Science gateway for molecular simulations Requirement: Workflow enabled grid portal for UNICORE based on Liferay Solution: – Adaptation of WS-PGRADE from GridSphere to Liferay – Development of UNICORE Submitter – Support for SAML – Development of ASM (Application Specific Module) support

23. MosGrid Science gateway - architecture

25. MosGrid Science gateway – application domains Quantum Chemical Calculations – based on approximated solutions of the Schrödinger equation – Average scalability – Gaussian, NWChem, Turbomole, … Molecular Dynamics – based on forcefields describing molecular interactions – Good scalability – Gromacs, NWChem, Amber, … Docking – Based on simplified forcefields – Excellent scalability – CADDSuite, FlexX, …

26. MosGrid Science gateway – application domains – molecular dynamics

27. SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481 Thank you for your attention! Any questions? http://www.sci-bus.eu/