1. Introduction to the Application Hosting Environment
Stefan Zasada

2. Contents Motivation for the AHE Concepts & functionality
Meeting the AHE design constraints
Architecture of the AHE
AHE client interaction
Deploying the AHE

3. Motivation for the AHE Problems with current middleware solutions:
Difficult for an end user to configure and/or install
Dependent on lots of supporting software also being installed
Require modified versions of common libraries
Require non-standard ports to be opened on firewall
Large footprint – memory/disk space

4. The Application Hosting Environment
Based on the idea of applications as web services
Lightweight hosting environment for running unmodified applications on grid resources (NGS, TeraGrid) and on local resources (departmental clusters)
Community model: expert user installs and configures an application and uses the AHE to share it with others
Simple clients with very limited dependencies
No intrusion onto target grid resources

5. Virtualizing Applications
Application Instance/Simulation is central entity; represented by a stateful WS-Resource. State properties include:
simulation owner
target grid resource
job ID
simulation input files and urls
simulation output files and urls
job status
Application exposed as web service
clients and end-users deal with applications as services and application instances, not jobs
higher level of abstraction: applications are first class entities in the architecture
applications may be multi-component (need several jobs to instantiate an application)
some applications may be interactive (e.g. steerable)
WSRF provides clean interface, web services communicate with resource services which allow data to be stored and retrieved

6. AHE Functionality Launch simulations on multiple grid resources
Single interface to monitor and manipulate all simulations launched on the various grid resource
Run simulations without manually having to stage files and GSISSH in
Retrieve files to local machine when simulation is done
Can use a combination of different clients – PDA, desktop GUI, command line

7. Accessing Resources NGS UK NGS Local UCL resources DEISA GridSAM/
Leeds
Manchester
Oxford
RAL
HPCx
UK NGS
NGS
GridSAM/
Globus
Local UCL resources
DEISA
GridSAM/
SGE
GridSAM/
UNICORE

8. AHE Design Constraints
Client does not have Globus installed locally
Client is NAT'd and firewalled
Client does not have to be a single machine
Client needs to be able to upload and download files but doesn’t have local installation of GridFTP
Client doesn’t maintain information on how to run the application
Client doesn’t care about changes to the backend resources

9. Meeting the Constraints
AHE Client behind firewall => polls server to update job state etc.
Uses intermediate filestaging area => GridFTP not installed
All application specific information for running simulations on the grid resource is maintained on a central service => user can switch clients etc.
Location of binary on grid resource configured on server => user doesn’t need to know
GridSAM provides interface to job queue

10. Layered Architecture of the AHE
AHE client installed on users machine - can be combination of GUI, command line and PDA
2) Client interacts with AHE constituent web services. Services include a registry of installed applications and a registry of previously instantiated applications. Each time an application is launched, a new stateful web service is created to represent the application instance. The user interacts with this web service to monitor, terminated the job etc
3) The AHE sends a Job Submission Description Language (JSDL) document to the GridSAM job submission service, representing the application run. Each machine that the AHE is submitting on requires one GridSAM instance for submission.
4) GridSAM submits the job to the machine’s DRM (currently GridSAM plug-ins exist for Grid Engine, Globus and Condor, and a UNICORE plug-in is under development by the DEISA middleware team).

11. Service Architecture of the AHE

12. AHE Server Implementation
WSRF::Lite => services developed in Perl
WebDAV server
GridSAM => Globus grid
=> Sun Grid Engine
=> Condor pool
=> Unicore
MyProxy
PostgreSQL database
Apache/Tomcat container

13. AHE Server Deployment The expert user must:
Set up container to host services:
Apache/WSRF::Lite or modified Tomcat/WSRF::Lite
Set up PostgreSQL database and WebDAV server
If not already running set up GridSAM instance for grid resource
Deploy and configure the AHE services in the container
OMII stack installer will do all of this automatically
Once deployed, any number of applications can be hosted

14. Hosting a New Application
Expert user must:
Install and configure application on all resources on which it is being shared
Create a JSDL template for the application (easily cloned from exiting template)
Add the application to the RMInfo.xml file
Run a script to reread the configuration
Documentation covers whole process of deploying AHE & applications on NGS and TeraGrid

15. Client Implementation
GUI & command line clients implemented in Java
Client allows user to:
Discover appropriate resources
Launch application
Monitor running jobs
Query registry of running jobs
Stage files to and from resource
Terminate jobs
GUI client implements application launching as a wizard

16. Client Extensibility
Plugins can be added to process application input files to automatically discover the input and output files that need to be staged
If no plugin is available then a default case will allow users to specify input and output files manually
Plugins implement AHEConfParser interface and follow specific naming convention
Plugin .class files dropped into plug-in directory and picked up by GUI/command line clients

17. AHE Client Deployment Deploying client is trivial for the end user:
User’s machine must have Java installed
User downloads and untars client package
Imports X.509 certificate into Java keystore using provided script
Configures client with endpoints of AHE services supplied by expert user
Ready to go!

18. Constructing workflows with the AHE
By calling command line clients from Perl script complex workflows can be achieved
Easily create chained or ensemble simulations
E.g. HIV equilibration protocol implemented by:
ahe-prepare  prepare a new simulation for the first step
ahe-start  start the step
ahe-monitor  poll until step complete
ahe-getoutput  download output files
repeat for next step

19. Current Deployed Applications
Currently hosting:
NAMD
LAMMPS
DL_POLY
LB3D
Gromacs
CHARMM
Plan to host:
Trubal
POLCOMS

20. Summary
The AHE provides a lightweight, easily deployable environment for running unmodified scientific applications on the grid and local resources
The AHE server is designed to be deployed by an expert user who uses it to share applications installed on grid resources
The client is easily installed by any end user, requiring no intervention by system/network administrators
By calling the command line clients from scripts, complex scientific workflows can be implemented

21. Any Questions? Released in OMII 3.2.0 http://www.omii.ac.uk/downloads/
RealityGrid web site:
NeSCForge:
Mailing list:
discuss

22. Exercise 1 Installing and configuring the AHE client Tasks: Outcome
Install the AHE client on your system
Set up a keystore containing your grid certificate
Configure the client with settings for UCL’s AHE server
Confirm that the client is installed and working
Outcome
Installed and configured AHE client

23. Exercise 2 Launching an Application on the NGS using the AHE Tasks:
Launch the sort application with the AHE GUI client
Launch the sort application with the AHE command line client
Manually specify input and output files for an application
Retrieve application output from NGS machine
Outcome
Successfully run applications on NGS machines