Holding slide prior to starting show

Lessons Learned from the GECEM Portal David Walker Cardiff University

17 July 2006WoCo9, Prescott, Arizona3 The GECEM Project GECEM = Grid-Enabled Computational ElectroMagnetics. Two-year project funded by the UK Department of Trade and Industry (DTI). Collaborators: –Cardiff University –University of Wales, Swansea –BAE SYSTEMS –Hewlett-Packard –Singapore Institute of High Performance Computing –Welsh e-Science Centre

17 July 2006WoCo9, Prescott, Arizona4 Main Project Objective To apply grid technologies to enable large-scale scientific and engineering research across a globally-distributed extended enterprise. The partners in the extended enterprise only partially trust one another. CEM was chosen as the application area, but our approach could also be applied to CFD and structural mechanics (for example).

17 July 2006WoCo9, Prescott, Arizona5 Subsidiary Project Objectives Explore secure sharing of code and simulation capability. Grid-enable legacy code by exposing software components as services. Develop a GECEM portal as an integrated user interface to the underlying GECEM services and tools.

17 July 2006WoCo9, Prescott, Arizona6 Aims of Talk Overview of GECEM portal. Lessons learned from the portal. Directions for future research.

17 July 2006WoCo9, Prescott, Arizona7 The GECEM Workflow GT3.2 services are used to create the surface and volume meshes, and to initiate the CEM simulation. The CEM simulation is migrated to a remote machine for execution. Simulation results are collaboratively visualised using the RAVE system. Geometry Create surface mesh Create volume mesh Perform CEM simulation Convert format Visualize

17 July 2006WoCo9, Prescott, Arizona8 GECEM Prototype Grid Geometry data Swansea Cardiff Other locations BAE SYSTEMS Create geometry Generate meshes CEM simulation Code and Meshes Output

17 July 2006WoCo9, Prescott, Arizona9 GECEM Production Grid Other locations Geometry data UWS Singapore BAE SYSTEMS Create geometry Generate mesh CEM simulation Code and Mesh Output

17 July 2006WoCo9, Prescott, Arizona10 GECEM Portal Based on GridSphere portal framework and GridPortlets Uses Tomcat Accesses GT3.2.1 services for mesh generation and CEM simulation migration. Accesses “vanilla” Web services for collaborative visualisation via RAVE. Uses JSR-168 compliant portlets to construct the portal.

17 July 2006WoCo9, Prescott, Arizona11 GECEM Portal Design CEM Migration Service Volume Mesh Service Surface Mesh Service RAVE Services MyProxy Server GridPortlets GECEM Portal UDDI registry GECEM Portlets

17 July 2006WoCo9, Prescott, Arizona12 GridPortlets Resource Browser Portlet –Resource Registry Credential Retrieval Portlet –Credential Management Service uses credential repository to store user credentials. File Browser Portlet Easy-to-use portlet for invoking Reliable File Transfer Service

17 July 2006WoCo9, Prescott, Arizona13 Main GECEM Portlets File Selection Portlet: user selects input data for mesh services and CEM migration service, and location of output. Based on GridPortlets File Browser Portlet. UDDI Portlet: user selects from a list of services. Service Invocation Portlet: invokes the service selected by UDDI Portlet using files from File Selection Portlet. RAVE Portlet: for collaborative visualisation.

17 July 2006WoCo9, Prescott, Arizona14 Other GECEM Portlets Job Status Portlet: view status of submitted jobs and input/output files. User can logout of portal and login again later to check progress. Progress Portlet: displays estimate of percentage of job completed. Progress Log Portlet: displays information on events happening as a job progresses. Conversion Portlet: converts ADF solver output file to RAVE-compatible format.

17 July 2006WoCo9, Prescott, Arizona15 RAVE Portlet RAVE supports remote rendering and collaborative visualisation, navigation and manipulation of data sets. Participants in a collaborative session can either explore data independently or in “follow-the-leader” mode. The RAVE Portlet allows a user to select machines to host the RAVE Data Service and perform the rendering, and to initiate or join a collaborative visualisation session.

17 July 2006WoCo9, Prescott, Arizona16 Security Makes use of certificates and Grid Security Infrastructure (GSI) from Globus project. Provides for single sign-on and credential delegation. Uses MyProxy Upload Tool embedded in portal to upload proxy certificate to online MyProxy repository. GridSphere’s Logon Portlet retrieves proxy certificate and stores it in the GridPortlets credential repository. Services authenticate users by interacting with the GridPortlets credential repository.

17 July 2006WoCo9, Prescott, Arizona17 GECEM Screenshots Login: Welcome: File selection: Service discovery: Job submission and status: Progress: RAVE:

17 July 2006WoCo9, Prescott, Arizona18 Positive Lessons Learned Portal provides a high-level user interface for scientists to access distributed resources easily. Portal shields users from the complexities of using the Grid. Users can access the portal from any web browser. Easy to integrate software from different sources into portal.

17 July 2006WoCo9, Prescott, Arizona19 Negative Lessons Learned Debugging distributed applications is hard! Portals built from portlets appear to be limited in their support for dynamic workflow composition. In the GECEM portal we have a static linear workflow in which each node is a placeholder for a service selected by the user from a list.

17 July 2006WoCo9, Prescott, Arizona20 Workflow Composition General-purpose service-based workflow composition environments are stand-alone, eg., Taverna, Triana, Kepler, rather than being web- or portlet-based. Scientific users appear to want more generality than the static workflows typified in GECEM. But they may not want the generality of a fully dynamic workflow composition environment.

17 July 2006WoCo9, Prescott, Arizona21 Proposed Solution Need a tool to create workflow structure by connecting together programming patterns or templates. For example, these 2 patterns: can be used to build this workflow:

17 July 2006WoCo9, Prescott, Arizona22 Proposed Solution (cont.) After the workflow structure has been created it is stored in a file. This file is then used to configure the portal. This approach acts as a portal builder for reasonably small, simple workflows.

17 July 2006WoCo9, Prescott, Arizona23 Other Issues In GECEM services are virtualised, but not data. Need a virtualised data store. Need to annotate intermediate and final data products with metadata which can then be used to perform searches across the data store. Need to move from GT3.2 services to GT4.x, or whatever.

17 July 2006WoCo9, Prescott, Arizona24 Conclusions Portals based on JSR-168 compliant portlets provide a good way of building user interfaces to distributed scientific applications. But tools are needed to allow new scientific workflows to be integrated into a portal.

17 July 2006WoCo9, Prescott, Arizona25 Web Sites GECEM: RAVE:

17 July 2006WoCo9, Prescott, Arizona26