1. SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481 CloudBroker Platform Presentation Wibke Sudholt CloudBroker GmbH Technoparkstrasse 1, CH-8005 Zurich, Switzerland Phone: +41 44 633 79 34 Email: wibke.sudholt@cloudbroker.com Web: http://www.cloudbroker.comwibke.sudholt@cloudbroker.comhttp://www.cloudbroker.com

2. © CloudBroker GmbH All rights reserved. CloudBroker Platform Web application store for the deployment and execution of scientific and technical software in the cloud On demand, pay per use, browser / programmatic / command-line access, cross-domain Uses infrastructure as a service (IaaS) from cloud providers Offers platform as a service (PaaS) for software vendors and software as a service (SaaS) for end users Easy to use, speeds up time to market, no need for own high performance computing infrastructure, OpEx instead of CapEx SCI-BUS Summer School 2012 2 05.07.2012 /

3. © CloudBroker GmbH All rights reserved. Platform Architecture SCI-BUS Summer School 2012 3 05.07.2012 / CloudBroker Platform Amazon Cloud IBM Cloud … Cloud Chemistry Appli- cations Biology Appli- cations Health Appli- cations Web Browser UI … Appli- cations REST Web Service API CloudBroker Integration User Tools End Users, Software Vendors CLI Java Client Library Engineering Appli- cations

4. © CloudBroker GmbH All rights reserved. Platform Functionality SCI-BUS Summer School 2012 4 05.07.2012 / Security Frame: Transport Layer Security, Access Rights Security IBM Cloud … Cloud Application Manager Application Manager Process Monitor Process Monitor Resource Manager Resource Manager Web Service API Web Browser UI Tools IBM Adapter IBM Adapter … Adapter Cloud Provider Access Manager Scalability and Fault Tolerance Handler User Manager User Manager Accounting Module Accounting Module Billing Module Billing Module Payment Module Payment Module Queuing System Queuing System Storage Manager Storage Manager Image Manager Image Manager Process Manager Process Manager Amazon Cloud Amazon Adapter Users

5. © CloudBroker GmbH All rights reserved. Platform Security SCI-BUS Summer School 2012 5 05.07.2012 / Security Certified Data Center CBP. Corporate IT Corporate IT SSL Secured Connection Authentication Industry Standard Server Security Technology CBP. Industry Standard Secure Data Center Security Certified Compute and Storage Cloud Technology Customer CloudBroker Cloud Provider SSL secured connection Corporate Security Policies and Standards Authentication to VM SSL Secured Connection Authentication to Cloud CloudBroker Platform Industry Standard Application Security Technology Cloud Instances Dedicated, Secured and Restricted Virtual Machines Client Browser or Application Client Browser or Application

6. © CloudBroker GmbH All rights reserved. User Management User roles: admin, advanced, standard Software visibility: private, protected, public SCI-BUS Summer School 2012 6 05.07.2012 /

7. © CloudBroker GmbH All rights reserved. Typical Job Submission Lifecycle 1.Prepayment (user) or monthly invoice (platform) 2.Software and resource selection and job creation (user) 3.Data file upload (user) to cloud storage (platform) 4.Job submission (user) 5.Compute instance startup or reuse (platform) 6.Data file upload from cloud storage to master node (platform) 7.Computations on worker nodes (platform, application) 8.Data file download from master node to cloud storage (platform) 9.Compute instance shutdown or reuse (platform) 10.Data file download (user) from cloud storage (platform) 11.Accounting and billing (platform) SCI-BUS Summer School 2012 7 05.07.2012 /

8. © CloudBroker GmbH All rights reserved. Dynamic Cluster in the Cloud Communication to and between nodes via SSH Instance information in dynamically generated node file (MPICH / MPICH2 / Open MPI format, more formats can be added upon request) File sharing via NFS can be enabled SCI-BUS Summer School 2012 8 05.07.2012 /

9. © CloudBroker GmbH All rights reserved. Application Requirements SCI-BUS Summer School 2012 9 05.07.2012 / ItemRequirements / possibilities TypeScientific and technical applications DomainAll LicenseOpen source, free, commercial CharacterCompute-intensive, batch-oriented InterfaceCommand line, non-interactive RuntimeHours, days, weeks Operating systemLinux ParallelizationSingle-threaded / single-core, multi-threaded / multi-core, embarrassingly parallel / high-throughput, parallel across nodes (via MPI or other methods) WorkflowsSingle jobs, pipeline jobs, parameter sweeps (via API)

10. © CloudBroker GmbH All rights reserved. Ported Applications SCI-BUS Summer School 2012 10 05.07.2012 / ApplicationDomainRemarks BLASTDNA and protein sequence alignmentAvailable in public platform AutoDockProtein-ligand dockingAvailable in public platform GromacsMolecular dynamics simulationsAvailable in public platform X! TandemMass spectrometry data matchingAvailable in public platform OpenFOAMComputational fluid dynamicsAvailable in public platform RosettaProtein modellingOnly with own license TransATComputational fluid dynamicsOnly with own license SiestaMaterial scienceOnly with own license T-CoffeeDNA and protein sequence alignment FoldXProtein modelling RMathematics and statistics More applications continuously to be added Applications can also be added by users

11. © CloudBroker GmbH All rights reserved. Typical Software Deployment Lifecycle 1.Generation of installation script (vendor) 2.Creation and configuration of software (vendor) 3.Adding of executables (vendor) 4.Adding of installation package (vendor), including installation script and software package, if needed 5.Deploying of software (vendor) 6.Checking of software and installation script (manager) 7.Generation of deployments (manager) 8.Running of special deployment jobs (platform) 9.Checking and activation of deployments (manager) 10.Activation of software (manager/vendor) 11.Publication of software (vendor) 12.Running of jobs (user) SCI-BUS Summer School 2012 11 05.07.2012 /

12. © CloudBroker GmbH All rights reserved. Clients and APIs SCI-BUS Summer School 2012 12 05.07.2012 / CLI Command line interface Job submission Java Application programming interface Java client library REST Representational state transfer Web service interface

13. © CloudBroker GmbH All rights reserved. Public Platform https://platform.cloudbroker.com First preview version published in fall 2010, current version 1.0.4 Available clouds: – Amazon EC2 and S3 (computing and storage) – IBM SmartCloud Enterprise (only computing currently, storage upcoming) Available applications: – BLAST, AutoDock, Gromacs, X! Tandem, OpenFOAM – Additional applications privately or upon request Software checking before deployment Standard pricing: – Cloud resource costs + software costs + CloudBroker surcharges Production version, monitored and maintained Help desk under platform@cloudbroker.complatform@cloudbroker.com Also available as hosted and in-house version SCI-BUS Summer School 2012 13 05.07.2012 /

14. © CloudBroker GmbH All rights reserved. Protein Modelling Example Case Setup: – Collaboration between IBM, CloudBroker and the Institute of Molecular Systems Biology (IMSB) at ETH Zurich – Goal is to better understand the mechanisms of infectious diseases – For the example of the Strepto- coccus bacterium, predict the 3D structures of important proteins from their 1D sequences by means of the Rosetta software – Use the CloudBroker Platform to manage the calculations on top of IBM SmartCloud Enterprise compute and storage resources Results: – 249 target proteins modeled – More than 1’000 CPUs used in parallel – 36’000 jobs automatically managed – Almost 250‘000 CPU hours consumed – 2.3 Mio 3D protein structure models created – Less than two weeks runtime SCI-BUS Summer School 2012 14 05.07.2012 /

15. © CloudBroker GmbH All rights reserved. Further Examples ASCOMP TransAT (http://www.ascomp.ch)http://www.ascomp.ch – Computational fluid dynamics – Collaboration with HSR – Microsoft Silverlight-based web browser UI KNIME (http://www.knime.org)http://www.knime.org – Konstanz Information Miner – Workflow framework – Computational chemistry example case SCI-BUS (http://www.sci-bus.eu)http://www.sci-bus.eu – SCIentific gateway Based User Support – EU FP7 project – 11+6 user communities from different domains SCI-BUS Summer School 2012 15 05.07.2012 /

16. © CloudBroker GmbH All rights reserved. CloudBroker in SCI-BUS SCI-BUS Summer School 2012 16 05.07.2012 /

17. © CloudBroker GmbH All rights reserved. Goals in SCI-BUS Provide connection to open and commercial cloud infrastructures for the scientific gateways Allow scientific gateways to charge for their services via the CloudBroker Platform Integrate CloudBroker Platform as new distributed computing infrastructure (DCI) in WS-PGRADE / gUSE SCI-BUS Summer School 2012 17 05.07.2012 /

18. © CloudBroker GmbH All rights reserved. Features to Be Implemented Already available: – Development and testing platform for SCI-BUS – Extended CloudBroker Platform REST and Java APIs – Integration of CloudBroker Platform in WS-PGRADE / gUSE under DCI Bridge, security and job settings – Possibility to configure and run CloudBroker jobs from within WS-PGRADE / gUSE workflows In preparation: – Commercial components in WS-PGRADE / gUSE for user mapping, billing and cost display – Possibility to register own cloud resources in the CloudBroker Platform – Adapters to open cloud infrastructures (OpenStack, OpenNebula, Eucalyptus) SCI-BUS Summer School 2012 18 05.07.2012 /

19. © CloudBroker GmbH All rights reserved. Acknowledgements CloudBroker GmbH: – Maxim Malgin, Ann Mikhailova ScaleTools AG: – Nicola Fantini, Andrey Sereda MTA SZTAKI: – Peter Kacsuk, Zoltan Farkas, Akos Hajnal … and many others SCI-BUS Summer School 2012 19 05.07.2012 /

20. SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481 Thank you! Questions? Please add yourself to the tutorial user list!