1. Shaowen Wang 1, 2, Yan Liu 1, 2, Nancy Wilkins-Diehr 3, Stuart Martin 4,5 1. CyberInfrastructure and Geospatial Information Laboratory (CIGI) Department of Geography 2. National Center for Supercomputing Applications (NCSA) University of Illinois at Urbana-Champaign 3. San Diego Supercomputer Center (SDSC) University of California at San Diego 4. Argonne National Laboratory 5. University of Chicago November 11, 2007 SimpleGrid Toolkit: Enabling Efficient Learning and Development of TeraGrid Science Gateway

2. 2 Purpose Simply the learning of science gateways Simply the learning of science gateways Expedite the prototyping process of developing science gateways Expedite the prototyping process of developing science gateways

3. 3 Background Grid computing Grid computing Science and engineering gateway Science and engineering gateway Problem solving environments (PSE) Problem solving environments (PSE)

4. 4 Related Work Active area Active area –Evidenced by TeraGrid Science Gateway activities Examples Examples –Gridport –Gridsphere Vine –OGCE

5. 5 The State of the Art Evolving and sophisticated web portal technologies Evolving and sophisticated web portal technologies –GridSphere –Liferay –Sakai –Jetspeed Missing simple, robust, and reusable interfaces between applications and portals Missing simple, robust, and reusable interfaces between applications and portals –Significant gap between Grid technologies and application problem solving environments Grid middleware complexity Grid middleware complexity –Grid technologies focus on enabling resource sharing and federation –The development of problem solving environments requires extensible, programmable, reusable, application-oriented software components that support customizable access to Grid and VO capabilities

6. 6 SimpleGrid Motivation Grid and web portal technologies are complex, and still rapidly evolving Grid and web portal technologies are complex, and still rapidly evolving An effort to close the gap between Grid computing and scientific applications An effort to close the gap between Grid computing and scientific applications

7. 7 SimpleGrid – Component- Based Design

8. 8 Architecture – External Interfaces

9. 9 Architecture – Internal Interactions

10. 10 Efficient Learning and Development Three-stage learning Three-stage learning – –Command-line – –Grid-enabled java application development – –Portlet development Simple installation and deployment Simple installation and deployment –Java, Ant, Tomcat, GridSphere –Globus Toolkit 4.0+ only for command-line stage Reusable components for development Reusable components for development –SimpleGrid APIs –JSP and Velocity templates Development environment setup Development environment setup –Manual for SimpleGrid setup in Eclipse

11. 11 From Individual to Community TeraGrid command-line tools for individual use SimpleGrid APIs to automate the access to cyberinfrastructure resources SimpleGrid portlets to enable community access to scientific problem solving capabilities as deployable components in science gateway portals

12. 12 SimpleGrid APIs SimpleCred: Grid proxy management SimpleCred: Grid proxy management SimpleTran: Data transfer to/from Grids SimpleTran: Data transfer to/from Grids SimpleRun: Grid job management SimpleRun: Grid job management SimpleViz: Visualization component SimpleViz: Visualization component SimpleInfo: Grid information provider SimpleInfo: Grid information provider –Under development –Current Grid information is provided statically through a configuration file

13. 13 SimpleCred Fetch Grid credentials Fetch Grid credentials –Local proxy loading or instantiation –Remote proxy instantiation through MyProxy Automatic credential renewal Automatic credential renewal –Simple interface for Grid proxy renewal, i.e., SimpleCred.get() Grid community user support Grid community user support –A global SimpleCred instance can be stored in portal as a shared object for users using the same community account Programming interface Programming interface –load(), logon(), get() Portlet interface Portlet interface –Grid credentials can be managed explicitly through a UserPortlet interface

14. 14 SimpleTran A wrapper of GridFTP A wrapper of GridFTP Threaded implementation Threaded implementation –Allow responsive interactions between portal and client browser

15. 15 SimpleRun A wrapper of GRAM and WS-GRAM A wrapper of GRAM and WS-GRAM –Support both GT2 and GT4 job submission –User selectable Depends on SimpleTran to transfer datasets Depends on SimpleTran to transfer datasets Programming interface Programming interface –execute() –getStatus()

16. 16 SimpleViz Visualization mechanisms Visualization mechanisms –JFreeChart –Google map –ParaView (under development) Threaded implementation Threaded implementation Portlet interface Portlet interface –Google map-based JavaScript library

17. 17 Portlet Components and Interfaces UserPortlet UserPortlet –User information and Grid credential management –Interface: JSP –Portlet: GridSphere ActionPortlet DMSPortlet DMSPortlet –A typical scientific computational analysis process –Interface: Velocity –Portlet: VelocityPortlet Portlet container Portlet container –GridSphere http://www.collab-ogce.org/ogce2/velocity-portlets.html

18. 18 Case Study Two-dimensional spatial interpolation in Geographic Information Systems Two-dimensional spatial interpolation in Geographic Information Systems Nearest-neighbor search procedure Nearest-neighbor search procedure –Computing intensive for large spatial datasets and/or high-resolution interpolation A fast two-dimensional spatial interpolation algorithm called DMS (Dynamically Memorized Search) A fast two-dimensional spatial interpolation algorithm called DMS (Dynamically Memorized Search) –Parameter-sweeping application for sensitivity analysis

19. 19 TeraGrid-Based DMS Analysis Request an individual or community account on TeraGrid Request an individual or community account on TeraGrid Install DMS executables on three TeraGrid sites Install DMS executables on three TeraGrid sites Prepare a dataset on a local machine Prepare a dataset on a local machine Transfer a specified dataset to a TeraGrid site (e.g., NCSA) Transfer a specified dataset to a TeraGrid site (e.g., NCSA) Submit a Grid job to the specified TeraGrid site with a parameter value Submit a Grid job to the specified TeraGrid site with a parameter value The submitted job is scheduled to be executed on one compute node on the specified TeraGrid cluster The submitted job is scheduled to be executed on one compute node on the specified TeraGrid cluster When the job is finished, the analysis result is written into the data directory of DMS installation on the TeraGrid cluster When the job is finished, the analysis result is written into the data directory of DMS installation on the TeraGrid cluster Transfer the result back to the local machine Transfer the result back to the local machine Visualize the result using the DMS visualization tool Visualize the result using the DMS visualization tool

20. 20 DMS Analysis Portlet

21. 21 Case Study Summary 16 participants 16 participants –various levels of software development experience and Grid computing knowledge 2.5 hours, all participants including those with minimum Java programming knowledge 2.5 hours, all participants including those with minimum Java programming knowledge –Master the SimpleGrid APIs for the DMS analysis –Successfully set up a portlet for the analysis in a GridSphere portal server

22. 22 Concluding Discussion The SimpleGrid toolkit The SimpleGrid toolkit –Makes an abstraction of generic Grid middleware services Enables science gateway developers to concentrate on developing PSE by working on reusable and extensible software components Enables science gateway developers to concentrate on developing PSE by working on reusable and extensible software components –Hides the complexity of evolving web portal technologies by tailoring to application requirements for developing PSE Service-oriented architecture Service-oriented architecture Component-based framework Component-based framework Simplify science gateway development Simplify science gateway development Help overcome the learning curve of science gateway technologies Help overcome the learning curve of science gateway technologies

23. 23 Ongoing Work APIs APIs –Grid-based visualization –SimpleInfo –Workflow Automation tools Automation tools –Enable automatic application integration as science gateway portal components (portlets) –User interface definition and generation –Workflow code stubs and Grid-related server-side code skeletons

24. 24 Acknowledgements CyberInfrastructure and Geospatial Information Laboratory (CIGI) CyberInfrastructure and Geospatial Information Laboratory (CIGI) National Center for Supercomputing Applications (NCSA) National Center for Supercomputing Applications (NCSA) NSF TeraGrid NSF TeraGrid

25. 25 Demo