1. Tomasz Haupt Northeast Parallel Architectures Center, Syracuse University For Computational Communities Constructing

2. Overview 4 What is a Web Portal? 4 Web Portal Architecture 4 Distributed Components: WebFlow 4 Interfaces: –Task Descriptor –Grid Interface 4 Portal Security 4 Summary

3. Customizable access to information and services What is a Web Portal?

4. Portal is not a static web page 4 It relies on a sophisticated browser technology –DHTML, JavaScript, cookies, applets, … 4 Server side processing –cgi-bin, servlets, asp, jsp, server side includes, XML –search engines, mail servers, calendar,... 4 Back End –data bases –credit card processing –external services: news, weather, stock quotes,...

5. Computational Portals 4 To provide a problem-oriented interface (a Web portal) to more effectively utilize HPC resources from the desktop via the Web browser. 4 This “point & click” view hides the underlying complexities and details of the HPC resources and creates a seamless interface between the user’s problem description on his/her desktop system and the heterogeneous computing resources 4 These HPC resources include supercomputers, mass storage systems, databases, workstation clusters, instruments, and visualization servers.

6. Example: Nanomaterials Research Gaussian Gamess convert data repository select edit QS Features: Data Flow computations, user supplied modules, seamless access to heterogeneous mixture of computational resources, seamless data transfer, visualizations, data management. Goals: automate the task, maximize throughput.

7. Example: LMS Landscape Management System WMS EDYSCASC2D DEM Land Use Soil Texture Vegetation EDYS: vegetation modelCASC2D: watershed model WMS: Watershed Modeling System Features: access to remote data (distributed databases, internet repositories), data pre- and postprocessing, tightly coupled applications running on remote hosts, visualizations Goal: decision support system available anytime, anywhere

8. Example: Gateway System Problem Solving Environment Features: access to remote data, data pre- and postprocessing, applications running on remote hosts, visualizations, archivization. Goal: guide the user to select software, generate input files, submit jobs, analyze data; hide complexity and details of a heterogeneous back end. Resources (software, hardware) templates, visualization tools Resource Allocation Problem Description newselectarch Input files Output files

9. Design Issues 4 Support for a seamless access (security) 4 Support for distributed, heterogeneous Back-End services (HPCC, DBMS, Internet,...) managed independently 4 Variable pool of resources: support for discovery and dynamical incorporation into the system 4 Scalable, extensible, low-maintenance Middle Tier 4 Web-based, extensible, customizable, self-adjusting to varying capacities and capabilities of clients (humans, software and hardware) Front End 4 Access to desktop applications

10. Towards the solution... problem description (physics, chemistry,...) Task description: I need 64 nodes of SP-2 at Argonne to run my MPI-based executable “a.out” you can find in “/tmp/users/haupt” on marylin.npac.syr.edu. In addition, I need any idle workstation with jdk1.1 installed. Make sure that the output of my a.out is transferred to that workstation Middle-Tier : map the user’s task description onto the resource specification; this may include resource discovery, and other services Resource Specification, Control Access, Events Resource Allocation: run, transfer data, run

11. Three Tier System Task Descriptor Resource Descriptor Front End: Tools to select or specify the problem to solve Middle Tier: Translates the user task into resource requests Back End: Resources and data to execute the task.

12. Abstract Application Descriptor (AAD) 4 “man pages written in XML” 4 specifies how to install and run the application on different hosts [current status of Gateway] 4 describes requirements, input and output data, options, arguments, etc. 4 to submit a job it must be reduced to a job descriptor (select host, options, input data…) More on AAD: http://www.npac.syr.edu/users/haupt/WebFlow/MODULES/AAD.html

13. Reducing AAD to JD Abstract Application Descriptor to Job Descriptor AAD select host JD select options select input... submit Generate batch script Generate RSL Information service (MDS) Resource broker JINI condor GUI Problem Solving Environment Data Flow manager

14. Example Job Descriptor (with selected application, host and i/o files) selected application selected host how to run it it expects this input file actual location of the file it generates this output file store it there save stdout and stderr

15. simple job object (atomic task) “input port”: method to be invoked “output port”: event fired run(); successfailure AAD

16. Complex Tasks run(); successfailure run(); successfailure run(); successfailure run(); successfailure run(); successfailure run(); successfailure

17. Task Descriptor 4 A computational task requested by the user may involve many steps. 4 Some steps can be performed concurrently, but typically there are data dependencies that force execution of the steps in some particular order. 4 Tasks can be defined recursively. 4 Task may specify resources explicitly, or provide requirements and/or preferences leaving the selection of resources to the discretion of a resource broker.

18. <!ATTLIST TaskName name CDATA #REQUIRED descriptor CDATA #IMPLIED> <!ATTLIST output task CDATA #REQUIRED event CDATA #IMPLIED> <!ATTLIST input task CDATA #REQUIRED method CDATA #IMPLIED> <!ATTLIST InputPort task CDATA #REQUIRED> <!ATTLIST OutputPort task CDATA #REQUIRED> ATD.dtd Example Task Descriptor

19. How the task descriptors are generated ? 4 Predefined (“set of scenarios”) 4 Created interactively by the user using Front End tools 4 Generated by middle-tier components

20. LMS Front End Navigate and choose an existing application to solve the problem at hand. Import all necessary data. Retrieve data Pre/post-processing Run simulations Select host Select model Set parameters Run

21. QS Front End Compose interactively your application from pre-existing modules Data-Flow Front-End

22. Building an application XML A visual representation is converted into a XML document XML service Web Server save parse ApplContext Generates Java code to add modules to ApplContext Publishes IOR Front-End Applet Middle-Tier

23. Gateway Front End Gateway Navigator where do you want to go today? Define the system you are interested in Control applet: File Access Job monitor

24. Middle-Tier: WebFlow Server CORBA-based distributed components WebFlow server is given by a hierarchy of containers (contexts) and components The server is the root context. A context knows its location in the hierarchy has attributes maintains a persistent state controls its children life-cycle is responsible for intercomponent communications (events) can be specialized by adding services (WebFlow modules) User 1User 2 Application 1 Application 2 App 2App 1 WebFlow Services

25. Distributed Middleware Master WebFlow Server Web Server Clients Distributed Back-End Resources Download Applet WebFlowContext Proxies Task Descriptor Grid Interface  JBDC  Information Services “slave” WebFlow Server

26. User Context Problem Context Example: Gateway Components Using PSE (Front End) user defines a problem Session is an instance of it (an attempt to solve it) Session comprises jobs Session context reflects the structure of ATD Session context can submit itself Session Context Job context application descriptor, job id, date submitted, completed, input file(s), output file(s)

27. WebFlow Events Method m Client B Event Adapter uses CORBA DSI,DII Event e A Context 1 Context 2 Module A does not care who is expecting the event; method fire Event invokes a method of its parent context Method m is a public method: anyone can invoke it, including the Event Adapter of Context 1. No protection against misuse! Dynamic Interfaces A B

28. Middle Tier Components Task Specification Resource Specification Component Container XML parser File access & transfer job control User Context profile Credentials (proxy) Session Context Job object Job object Job object batch script generator resource broker data flow manager NetSolve Linear Algebra proxy PSE support Information services database access data analysis Multi- disciplinary task control archivization access control context lifecycle

29. Grid Interface 4 How to hide complexity and details of Back End resources? 4 Example: JDBC Servlet Application Driver Manager Oracle D.Sysbase D.mSQL D. OracleSysbase mSQL Front End Back-End independent business logic Java.sql Middle-Tier

30. JDBC model to provide access to computational resources Servlet Application GRAM PBS NQS CONDOR O2KSP2NOW Front End Back-End independent business logic Grid Interface Grid Interface: access control, allocations, resource look-up, discovery, (co)allocation, monitoring, QoS, fault tolerance, services, events,... Addressed by Grid Forum. Approximated by Globus. Portal: builds on top of it, implementing proxies

31. Example of a proxy module &(rsl_substitution = (MYDIR “/tmp/haupt”)) (DATADIR $(MYDIR)/data) (EXECDIR) $MYDIR)/bin)) (executable = $(EXECDIR)/a.out) (arguments=$(DATADIR)/file1) (stdout=(MYDIR)/result.dat)) (count=1) GRAM resource description Generate Data Run Job Analyze The Run Job module is a proxy module. It generates the RSL in-the-fly and submits the job for execution using globusrun function. The module has access to a job descriptor.

32. Security: Issues Front End (Applet or Application) Connection through open Internet access control Gatekeeper HPCC resources Layer 1: Secure Web Layer 2: Secure Middle Tier Layer 3: Secure access to resources Policies defined by resource owners access control and delegation The same of different security domain

33. Security (2) 4 Different model than most commercial solutions: –charge for service and not CPU used to render the service –identify by credit card number 4 Three-tier architecture: –delegation of credentials

34. Security: CORBA security service

35. Kerberos/SecurID Web Server ORB Master WebFlow Server ORB Slave WebFlow Server SECIOP krsh C:\>kinit C:\>krsh (with forwardable ticket) Download frameset applet Front End Middle Tier Back End

36. SSL Web Server ORB Master WebFlow Server ORB Slave WebFlow Server https Globus GSSAPI Download frameset applet Front End Middle Tier Back End IIOP Servlets

37. Proxy Objects 4 The master creates and maintains proxies for each component – to forward requests from the Web client to remote objects –Simplify the association of the distributed components –Enable the communication between the client and the slave servers running on different hosts –having the capability of logging, tracking and filtering all messages between components in the system to implement fault tolerance and security and transaction monitors.

38. Distributed Middleware Master WebFlow Server Web Server Clients Distributed Back-End Resources Download Applet WebFlowContext Proxies Task Descriptor Grid Interface  JBDC  Information Services “slave” WebFlow Server

39. Clients and their servers Middle Tier Custom Servers Back End Servers and their services Emerging Object Web Multi-Server Model

40. Summary 4 We build Web Portals using new, emerging, often ephemeral technologies and standards 4 What will survive? –Multitier architectures –Distributed Components –XML to define interfaces –Metadata (UML, XMI, …) 4 What next? –Developer tools for enterprise servers –ASP: Application Service Providers

41. Summary (2) 4 Academic example: WebFlow –Gateway, LMS, GEM, NCSA 4 We extended notion of a Web Portal –support for HPCC –Grid Interface (a new CORBA facility?) –Abstract Task Descriptor in XML 4 Middle-Tier components as proxies 4 To be added: proxy communication channels