ANR CIGC LEGO (ANR-CICG-05-11) Bordeaux, 2006, December 11 th Automatic Application Deployment on Grids Landry Breuil, Boris Daix, Sébastien Lacour, Christian Pérez PARIS Research Team INRIA/IRISA Rennes

How to Deploy my Application on Grid Resources? Visualization Homogeneous cluster SAN Homogeneous cluster LAN WAN Supercomputer Flow code Transport code Controller velocity scalars concentration

Manual Deployment  Discover available grid resources  Select grid resources for execution  OS, architecture compatibility  Map the application onto the selected resources  MPI processes  Components  Select compatible compiled executables  Upload and install executables, stage input files in  Launch processes on remote computers  Set configuration parameters of the application  Components' attributes  Network topology information Too complex!

Automatic Deployment  Automatic  Resource discovery  Execution node selection  File installation  Process launch  Application configuration  Hide application complexity  Hide grid complexity stop reading your e- mails! Grid

Generic Application Description  Translator  From specific to generic application description  Straightforward to write CCM Appl. Description MPI Appl. Description GridCCM Appl. Description CCM Appl. Description MPI Appl. Description GridCCM Appl. Description CCM Planner MPI Planner GridCCM Planner Plan Execution Generic Application Description Deployment Planner Deployment Plan Execution

Identification of the Steps of Automatic Deployment MPI Application DescriptionCCM Application Description Resource DescriptionGeneric Application DescriptionControl Parameters Deployment Planning Deployment Plan Execution Application Configuration Static Applications Deployment Tool

ADAGE Feature List  MPI, JXTA, CCM application descriptions  Network topology description  Generic application description  Simple control parameters, simple planner  Deployment plan execution (RSH/SSH)  Basic file transfer support  Aware of file visibility (~NFS)  Does not check for already available files  Redeployment support  Either re-play or add/remove elements

How Easy is it, in Practice?  As simple as A-B-C  adage-deploy–resource –application my_appl.zip –ctrl_param my_control_parameters  Grid resource description  Written once for all by grid admins  Optional control_parameters constraints  Keep control on the deployment process

ADAGE & Grid5000  ADAGE is not able to directly talk with OAR  PERL script to automatically generate resource description from OARgrid id  oarstat2resources.pl –g 6332 –outputfile r.xml  adage-deploy –inres r.xml …

Some ADAGE results  JXTA  1 peer == 1 process  M. Jan deployed  ~ peers on ~ 145 machines in ~ 95 seconds Most of the time take by ssh commands  CCM  1 component == 1 process  H. Bouziane deployed  4000 components on 974 processors machines 432 machines on several G5K sites  Enable to CCM-plugin optimization

Towards ADAGE v2  Complete rewrite of ADAGE  By L. Breuil  Objectives of ADAGE v2  Provide a clean code architecture  Ease the additions of  Programming model (GridCCM), ie plugins  Planner  Better back-end support  GAT, Taktuk, …  Support of “dynamic” applications  In cooperation with B. Daix, PhD

Discussion  ADAGE: a model to support automatic application deployment  Ongoing/Future work  Finish ADAGE v2 (~ begining 2007 )  Stabilize plugins API  Stabilize planner API wrt to resources  Dynamic application management  PadicoTM support?  Adapt application description in function of resources  DIET support  Not directly targeted  Fault tolerance  Application monitoring