Convert generic gUSE Portal into a science gateway Akos Balasko

Outline ● General Session: ● Goal – Terminology – Basic Idea – Concept of ASM ● Hands-On 2

Goal Community members want to: Execute scientific applications Using distributed systems to: decrease time cost Exploit storage & computational capacities use complex systems totally hidden Do not want to : learn any techniques that don't fit with scientific area of them → would like to use web-interfaces based on gUSE

Examples HP-SEE Portal for bioinformatics Small scale launch (Home cluster): PBS/Linux Cluster, at the Obuda University – John von Neumann Faculty of Informatics. Activity and technical assistance in pre-production stage: Technical assistance was provided by MTA SZTAKI and NIIF. Porting: Application was ported using(Perl/C). Workflow and GUI was created for the application by Obuda University. Benchmarking Scaled from 32 cores to 96 cores (MPI). DeepAligner Status The online service is using two from NIIF’s supercomputing infrastructure (Budapest site and Szeged site). More information:

Where are we now? 5 ASM API WS-PGRADE UI Customized UI Other, existing UI WS-PGRADE workflow UI

Terminology Grid Application Developer – would like to have a Portal customized for the application and end user community – knows how to develop a grid application in gUSE/ws-PGrade or how to port a legacy application to the grid with gUSE/ws- PGrade Grid Portal Developer – He knows the Java language and JSP (Java Server Pages) or other techniques (IceFaces etc) – He does not need to know the source code of gUSE – He and also the end users have user certificates to access the grid

Grid Application Developer: – Develops a new application – App. must be offered for others through a Web-based portal → Portal is highly customised for the application and the user community. People: – Do not know grids – Should be able to execute the grid application with their own custom inputs – Using the Web portal. Application specific portal Grid application developer End users Goal

Terminology Grid Application Developer Develops a grid application with gUSE Exports it to the local repository of gUSE End Users, Scientists execute a copy of the exported application with custom input parameters Grid Portal Developer Develops an interface (in general a portlet) that fits to the application I/O pattern and the end users’ needs and associate it with the role Connects the GUI to gUSE using the programming API of Application Specific Module(ASM)

Basic idea A new module called “Application Specific Module” has been developed for gUSE that ease the creation of Application Specified Portlets –The module provides programming API to control applications developed using gUSE –The module provides programming API to access inner gUSE services Any kind of Graphical interface can be developed on top of this module –GUI written in JSP, JSTL (Java Servlet Tag Library ) and Java –GUI can be customized for the grid application –GUI can be customized for end users

Concept ws-PGrade Internal components submitter Desktop or Service grids / cloud / Clusters DCIs gUSE Web browser Client 10

Concept ws-PGrade Internal components submitter Desktop or Service grids / cloud / Clusters DCIs gUSE Web browser Client Developed web-interface 11 We will focus on this!

Concept in details Web browser Client Developed web- interface Web pages (JSP, JSF or other) JAVA event handler, and inner classes Application Specific Module API Configuration files (wsdd) You develop it 12

Concept ws-PGrade Internal components submitter Desktop or Service grids / cloud / Clusters DCIs gUSE Web browser Client Developed web-interface 13 Placing web-app as gUSE component

DCI systems gUSE ws-PGrade Submitter Workflow Interpreter Repository Workflow Storage Storage Information User space Concept in details 14

DCI systems Grid Application Developer creates own Grid Application via ws-PGrade and export it to the local repository gUSE Repository ws-PGrade Grid Application Submitter Workflow Interpreter Workflow Storage Storage Information User space Concept in details 15

gUSE DCI systems Then the Grid Portal Developer creates a new web-application (mainly an web-interface) which fits to the user community's needs, and connects it to the gUSE using ASM API ws-PGrade Developed web- interface ASM Repository Grid Application Submitter Workflow Interpreter Workflow Storage Storage Information User space Concept in details 16

gUSE DCI systems Finally the end-users create own application from the exported one, parameterize and execute it using the developed portlet ws-PGrade Developed web- interface ASM Repository Submitter Workflow Interpreter Workflow Storage Storage Information User space Grid Application Concept in details > 17

gUSE DCI systems Finally the end-users create own application from the exported one, parameterize and execute it using the developed portlet ws-PGrade Developed web- interface ASM Repository Submitter Workflow Interpreter Workflow Storage Storage Information User space Grid Application Concept in details 18

gUSE DCI systems Finally the end-users create own application from the exported one, parameterize and execute it using the developed portlet ws-PGrade Developed web- interface ASM Repository Grid Application Submitter Workflow Interpreter Workflow Storage Storage Information User space Concept in details 19

gUSE DCI systems Finally the end-users create own application from the exported one, parameterize and execute it using the developed portlet ws-PGrade Developed web- interface ASM Repository Submitter Workflow Interpreter Workflow Storage Storage Information User space Grid Application Concept in details 20

gUSE DCI systems Then get back the results ws-PGrade Developed web- interface ASM Repository Submitter Workflow Interpreter Workflow Storage Storage Information User space Grid Application Concept in details 21

AutoDock workflow ● Run docking simulations ● AutoGrid: – Role: generator (n job instances) – Creates inputs for AutoDock ● AutoDock: – Runs as a PS job – Performs the docking simulations ● Collector: – Examines the results – Collects m best results Inputs to be uploaded receptor.pdb placed in port 0 Number of job instances to be created as Command Line Argument Output placed to port 1 of Collector job

Introduction Would like to have : 23

Access to development framework ● Login to your cloud image: – ssh -Y – (here you will have a pre-installed WS-PGrade/gUSE 3.5.2) ● Start netbeans: –./netbeans-6.5.1/bin/netbeans – Open AutoDock webapplication(left handside, projects panel), – then navigate to Source Packages – hu.sztaki.lpds.autodock package – and finally click AutoDockPortlet.java for edition 24

Implement method for workflow import //// EXCERCISE 1: Import a workflow, implement doImport method ### //// // Predefinitely we know on what prefix used for the exported workflow name String workflowPrefix = "autoDock_" + userId; // Predefinitely we know who exported the workflow String developerID = "11004"; // get the type of the workflow exported (it's been exported as Application) String impWfType = RepositoryItemTypeConstants.Application; // get the instance of ASMService singleton ASMService asmService = ASMService.getInstance(); // get the workflow ID to be imported Vector list = asmService.getWorkflowsFromRepository(developerID, impWfType); // it returns a list, grab the last one, the list is ordered according to time ASMRepositoryItemBean workflowToImport= list.lastElement(); // invoke ImportWorkflow method with the parameters asmService.ImportWorkflow(userId, workflowPrefix, developerID, impWfType, workflowToImport.getId().toString()); 25

Set command line argument //// ### EXCERCISE 2: Set command line arguments to the workflow ### //// // get ASMService instance ASMService asmService = ASMService.getInstance(); // get name of the job (its commandline arg. will be set) String jobName = "AutoGrid"; // invoke setcommandLineArg method with the given parameters asmService.setCommandLineArg(userID, workflowName, jobName, workUnitNumb); 26

Upload an input file //// ### EXCERCISE 3.: Upload and associate inputs and command line interfaces ### //// // get instance of ASMService singleton ASMService asmService = ASMService.getInstance(); // get the name of the job which has the input file to be set String jobName = "AutoGrid"; // upload the file to the portal server once File uploadedFile = asmService.uploadFiletoPortalServer(file,userId,fileName); // than upload it to the storage and associate it to the given job's given port asmService.placeUploadedFile(userId, uploadedFile, workflowName, jobName, portNumber); 27

Implement a method to submit a workflow Implement doSubmit method (can be found by searching for Excercise 4 in the code) try { // get userID from ActionRequest object String userID = (String) request.getRemoteUser(); // get the ASMService instance ASMService asmService = ASMService.getInstance(); String workflowName = request.getParameter("selectedWorkflow"); // notifytext and notifyType can be added as empty strings. String notifyText = ""; String notifyType = ""; // invoke its submit function with the parameters asmService.submit(userID, workflowName, notifyText, notifyType); // catch all exceptions } catch (ClassNotFoundException ex) { Logger.getLogger(AutoDockPortlet.class.getName()).log(Level.SEVERE, null, ex); } catch (InstantiationException ex) { Logger.getLogger(AutoDockPortlet.class.getName()).log(Level.SEVERE, null, ex); } catch (IllegalAccessException ex) { Logger.getLogger(AutoDockPortlet.class.getName()).log(Level.SEVERE, null, ex); } 28

Implement a method to delete a workflow //// ### Excercise 5: Implement doDelete method ### //// // get userID from ActionRequest object String userID = (String) request.getRemoteUser(); // find and get the name of the selected workflow among the parameters String selected_wf = request.getParameter("selectedWorkflow"); // get the instance of the ASMService Singleton ASMService asmService = ASMService.getInstance(); // invoke its DeleteWorkflow method with the parameters asmService.DeleteWorkflow(userID, selected_wf);

Deploy and execute → Register :Check and follow video ( → Restart the portal (needed if we add completely new component(e.g. ASM-based) to gUSE) – stop.sh –./guse/apache-tomcat /bin/startup.sh Note: DO NOT use start.sh!! – Check starting logs: tailf./guse/apache-tomcat /logs/catalina.out – Deploy your compiled web-application Login as (pass : Go to “Go to ” link, then select Control panel Scroll down, on the left click to plugin installation button Click to install more portlets button Then upload File link, finally select your war file (placed in the /guse/tutorial/Autodock/dist folder) – Initialize the portal with the wizard. Mysql server: replace IP to localhost MySQL username/pass : guse/guse Textboxes ot SMTP issues can be left empty – The portlet will be shown under ASM Tutorial menuitem (input file for uploading is placed in /guse/tutorial/Autodock/test folder)

End of Hands-On Thanks for your attention! Questions?