1. int.eu.grid: A grid infrastructure for interactive applications Gonçalo Borges LIP on behalf of Int.EU.Grid Collaboration INGRID’08, Italy, April 2008

2. 2 Do you really need to be motivated?  What is interactivity? Feedback channel between you and your application Interactivity is strongly connected to visualization  Why interactivity? Users want answers in seconds and not hours Grid debug, check application evolution,...  Is it possible to have it on the grid? Yes it is!!! Although the several middleware layers and the crossover of multiple administrative domains.  Benifit for you as a grid user? Power of the grid available at your fingertips!

3. INGRID’08, Italy, April 2008 3 REMOTE SITE Internet REMOTE SITE Middleware SERVICES Middleware Interactive Job Execution F1F2 Job F1F2 Job  Fast start-up  Execution in high-occupancy situations

4. INGRID’08, Italy, April 2008 4 Interactivity on the Grid  VOs and users are anxious to get it Needed by a wide set of applications in different scientific domains  Interactivity has been neglected by larger Grid projects… Standard grids are aimed to sequential jobs… How to properly set “Matchmaking” and “Brokering” for interactive tasks? How to start the application immediately? Even in scenarios in which all computing resources might be running batch jobs? And what about online application input/output? Int.EU.Grid Provide an advanced grid empowered infrastructure for scientific computing targeted to support demanding interactive (and parallel) applications.

5. INGRID’08, Italy, April 2008 5 Interactivity and MPI on the Grid  VOs and users are anxious to get it Needed by a wide set of applications in different scientific domains  MPI has been neglected by larger Grid projects… Aimed to sequential jobs… How to properly set “Matchmaking” and “Brokering” for parallel tasks on a Grid Environment? How to manage/control local cluster MPI support? How to set central MPI support? Int.EU.Grid Provide an advanced grid empowered infrastructure for scientific computing targeted to support demanding interactive and parallel applications.

6. INGRID’08, Italy, April 2008 6 Int.EU.Grid grid infrastructure  12 sites, 7 counties 9 in production 4 in development  ~ 900 COREs Xeon Opteron Pentium  ~ 45 TB of storage space  Interconnection by Geant

7. INGRID’08, Italy, April 2008 7 Int.EU.Grid grid infrastructure  Two sets of sites Production 9 sites Development 4 sites DEVELOPMENT PRODUCTION

8. INGRID’08, Italy, April 2008 8 Int.EU.Grid services  Distributed services Taking advantage of the partners expertise Redundancy Better use of resources Production Core Services CrossBroker RAS BDII VOMS LFC MyProxy Production Core Services CrossBroker RAS BDII VOMS LFC MyProxy APEL accounting GridICE R-GMA Development Core Services CrossBroker RAS BDII VOMS LFC MyProxy Pure gLite WMS Autobuild Repository R-GMA for development Helpdesk SAM Network monitoring Security coordination

9. INGRID’08, Italy, April 2008 9 Int.EU.Grid Virtual Organizations  Applications ifusion ienvmod iusct ibrain ihep iplanck iwien2k icompchem imrt euforia ihidra icesga imain, imon, itut, itest

10. INGRID’08, Italy, April 2008 10 int.eu.grid Arch Local Services Core Services Public ips I2G CB Top-BDII I2G RAS R-GMA Server LFC VOMS Server MyProxy I2G UI LCG-CE I2G CE software Batch Server I2G WN software gLite WN I2G WN software gLite WN I2G WN software gLite WN MonBox Site-BDII EGEE Node EGEE Nodes with I2G software I2G Node Legend: SE Public/Private ips  int.eu.grid is a gLite based infrastructure but... enhancing several services (lcg-CE, WN) and deploying new components (CB, RAS, UI) towards interactivity and MPI

11. INGRID’08, Italy, April 2008 11 Migrating Desktop and RAS (I)  Migrating Desktop (MD): User Friendly Grid Access Java based GUI; Hides the details of the grid Allows to log-in in the GRID independently from where you are (laptop, desktop, everywhere...) what kind of Computer/OS you are using…  Roaming Access Server (RAS): Gateway for Grid Access Performs actions on the grid on behalf of the MD UIRASMD GUI to access Grid Services Data Management Interactivity and Visualization Submission of Parallel Jobs Submission of Interactive Jobs

12. INGRID’08, Italy, April 2008 12 Migrating Desktop and RAS (II)  Work together in a client/server fashion The RAS is a core service… … installed at a central location with direct network connectivity MD can run from private workstations… … as long as it connects to the appropriate RAS ports  The RAS offers a well defined set of web services Represents the integration level of interfaces for different middleware Java 1.6 support Job submission, job monitoring, data management operations, job channel forwarding, …  MD/RAS supports interactivity and real time visualisation A plugin has to be developed for each application Acts as an interface enabling bidirectional communication between a job and the local GUI.

13. INGRID’08, Italy, April 2008 13 (i2)glogin ::== grid interactive tool  Interactivity is wrapped around (i2)glogin CrossGrid tool extended in int.eu.grid to execute with GT4 Supports GSS-based encryption Enables online communication providing shell functionality for access to the Grid nodes Sort of (grid) ssh using certificates  MD/RAS integration with i2glogin A local instance of i2glogin is started in the RAS Acting as the server The client part of i2glogin is submitted to the grid through a JDL The connection point (server:port) is specified as an argument When the grid job connects, i2glogin connects back to the local instance Creates a secure, low latency, bidirectional connection.

14. INGRID’08, Italy, April 2008 14 I/O streaming $ i2g-job-submit interactive.jdl $ i2glogin -p 24599:158.109.65.149 VirtualOrganisation = "imain"; JobType = “Normal"; Interactive = TRUE; InteractiveAgent = “i2glogin“; InteractiveAgentArguments = “-r –p 24599:158.109.65.149 -c“; Executable = “/bin/sh"; InputSandbox = {“i2glogin"};

15. INGRID’08, Italy, April 2008 15 I/O streaming CrossBroker WN User applicationi2glogin $ i2glogin -p 24599:158.109.65.149 Job

16. INGRID’08, Italy, April 2008 16 WN i2glogin I/O streaming CrossBroker User application $ i2glogin -p 24599:158.109.65.149

17. INGRID’08, Italy, April 2008 17 WN i2glogin I/O streaming CrossBroker User application $ i2glogin -p 24599:158.109.65.149 sh-2.05b$ hostname aow5grid.uab.es sh-2.05b$ exit exit Connection closed by foreign host $

18. INGRID’08, Italy, April 2008 18 GVID ::== Grid Video streaming  Visualisation capabilities are based on the compression of the OpenGL graphics generated by the application GVID performs the encoding/decoding on both sides Video encoding saves bandwidth Communication over glogin  A GVID display client java implementation was develloped to follow requirements of MD/RAS application plugins A generic java package is ready to be used by all grid devellopers trying to built a MD plugin User inputs (such as mouse clicks) generated through the application specific plugin are sent via RAS to the application The application has to be adapted to react to these input events

19. INGRID’08, Italy, April 2008 19 Integration of the Interactivity & Visualization tools (I) Aplication Layer Int.eu.grid middleware extension for visualisation Int.eu.grid lower middleware extension for interactivity

20. INGRID’08, Italy, April 2008 20 Integration of the Interactivity & Visualization tools Job Submission Services CrossBroker Logging & Bookkeeping Roaming Access Server LRMS Gatekeeper JDL, WorkerNode gLogin vtk App MP4 Encoder Event Decoder TCP Transport Socket Transport File Transport Pipe Transport remoteGlut Job Submission Services CrossBroker Logging & Bookkeeping LRMS Gatekeeper JDL,, WorkerNode gLogin WorkerNode vtk App MP4 Encoder Event Decoder TCP Transport Socket Transport File Transport Pipe Transport remoteGlut MP4 Encoder Event Decoder TCP Transport Socket Transport File Transport Pipe Transport remoteGlut Application GVid Migrating Desktop Java Video Player MP4 Decoder Event Encoder TCP Transport Socket Transport File Transport Pipe Transport Migrating Desktop Simulation +visualisation Java Video Player MP4 Decoder Event Encoder TCP Transport Socket Transport File Transport Pipe Transport Java Video Player MP4 Decoder Event Encoder TCP Transport Socket Transport File Transport Pipe Transport Java Video Player MP4 Decoder Event Encoder TCP Transport Socket Transport File Transport Pipe Transport Java Video Player MP4 Decoder Event Encoder TCP Transport Socket Transport File Transport Pipe Transport

21. INGRID’08, Italy, April 2008 21 CrossBroker (I)  CrossBroker: Int.EU.Grid meta-scheduler Offers the same functionalities as the EGEE Resource Broker, plus: Support for Interactive Applications Interactive agent injection Scheduling priorities; Time Sharing Full support for Parallel Applications PACX-MPI and OpenMPI Flexible MPI job startup based on MPI-START PACX-MPI and OpenMPI support MPI job startup based on MPI-START Support for interactivity via GVid/Glogin RBCrossBroker

22. INGRID’08, Italy, April 2008 22 CrossBroker (II)  The user can decide which interactive agent to use through special JDL requirements The CrossBroker can inject it transparently to the user  If the job is recognized as interactive... The CrossBroker treats it with higher priority  There is in place a mechanism to use bandwidth measurements in the matchmaking process Great for application needing visualisation But not really implemented...  If there are no available resources Use a time sharing mechanism

23. INGRID’08, Italy, April 2008 23 Time Sharing: Glide-in mechanism  Main idea: Wrap every batch job with an agent (glide-in) Agent will get control of the remote machine independently of its local resource manager.  Glide-in benefits on the interactivity framework Agents enable simple multiprogramming between interactive and batch jobs. Interactive jobs may run even when no free resources are available. Agents can also be used as a fast start-up mechanism. Agent can control the amount of CPU that an interactive job gets according to QoS requirements expressed by the user in the JDL.

24. INGRID’08, Italy, April 2008 24 Time Sharing Scheduling Agent Condor-G CrossBroker Grid Resource LRMS BATCH JOB

25. INGRID’08, Italy, April 2008 25 Time Sharing Scheduling Agent Condor-G CrossBroker Application Launcher Grid Resource LRMS BATCH JOB

26. INGRID’08, Italy, April 2008 26 Time Sharing Scheduling Agent Condor-G CrossBroker Application Launcher Grid Resource LRMS Agent VM1VM2 BATCH JOB

27. INGRID’08, Italy, April 2008 27 Time Sharing Scheduling Agent Condor-G CrossBroker Application Launcher Grid Resource LRMS Agent VM1VM2 BATCH JOB

28. INGRID’08, Italy, April 2008 28 Time Sharing Scheduling Agent Condor-G CrossBroker Application Launcher Grid Resource LRMS Agent VM1VM2 BATCH JOB Batch Job running

29. INGRID’08, Italy, April 2008 29 Time Sharing Scheduling Agent Condor-G CrossBroker Application Launcher Grid Resource LRMS Agent VM1VM2 BATCH INT. JOB

30. INGRID’08, Italy, April 2008 30 Time Sharing Scheduling Agent Condor-G CrossBroker Application Launcher Grid Resource LRMS Agent VM1VM2 BATCH INT. JOB Startup-time reduction Only one layer involved

31. INGRID’08, Italy, April 2008 31 Response Time Mechanism Resource Searching Resource Selection Submission Campus GridRemote Site Free machine submission 3s0.5s17.2s22.3s Glidein submission to free machine 3s0.5s29.3s33.25s Virtual Machine submission 0.5s6.79s8.12s CrossBrokerCE + WN

32. INGRID’08, Italy, April 2008 32 Local configurations  Address the possibility of starting up the application immediately  Local Resource Management System level Some sites have configured their queues so that (Interactive) jobs do not remain in queue Or there is immediate resources to execute them or they are resubmitted elsewhere  LCG-CE JobManager (JM) level Some site have decided to move from the “lcgpbs” JM to the “pbs” one In EGEE, a direct submission to a site completes in ~25s if the sites uses “pbs” JM against > 2m for the “lcgpbs” JM “pbs” JM requires sharing homes between CE and WN (via NFS) However, lcgpbs have been coded for scalability instead speading Less CPU consumption when the CE is handling hundred of jobs in parallel

33. INGRID’08, Italy, April 2008 33 EGEE short deadtime jobs (sdj) WG  sdj definition: short execution time; unexpected and urgent cannot be dealt on a best effort basis in a full production regime.  Use case & Requirements Visualisation, Inspection, Interaction Immediate access and small latencies Dinamic attach a sheel to a running process Debugging process  Bypassing middleware delays “An alternative solution is “agent scheduling” or “overlay” systems which provide user task management”  int.eu.grid relies on gLite with additional enhancements Opportunity window to test interoperability with EGEE

34. INGRID’08, Italy, April 2008 34 Scheme: Int.EU.Grid/EGEE shared WN I2G CrossBroker LFC REGISTRY Top-BDII lcg-RB Top-BDII LFCREGISTRY Int.EU.Grid Infrastructure EGEE Infrastructure I2G UI Batch Server MonBoxSite_BDII LCG-CE I2G CE software I2G WN software gLite WN I2G WN software gLite WN I2G WN software gLite WN LCG-CEUIMonBox Site-BDII SE Core Services Local Services EGEE Node EGEE Nodes with I2G software I2G Node Legend: Migrating Desktop JM env vars MPI, Visualization Int.EU.Grid & EGEE VOs

35. INGRID’08, Italy, April 2008 35 gLite WN Scheme: Enhancing an EGEE site with Int.EU.Grid features I2G CrossBroker LFC REGISTRY Top-BDII lcg-RB Top-BDII LFCREGISTRY Int.EU.Grid Infrastructure EGEE Infrastructure Core Services Batch Server LCG-CE gLite WN I2G WN software gLite WN I2G WN software MonBox Site-BDII SE UI Local Services I2G UI LCG-CE I2G CE software MonBox Site-BDII Batch Server I2G WN software gLite WN I2G WN software gLite WN I2G WN software gLite WN SE MPI, Visualization Migrating Desktop I2G UI software I2G CE software I2G WN software UI gLite WN LCG-CE MPI, Visualization

36. INGRID’08, Italy, April 2008 36 Open Issues and Drawbacks  Information System is still a major problem Delay reporting the site change of availability to Core Services  Still missing a generalized priority schema for grid users and jobs Things are more easy if infrastructure is simple, but… institutions will share the same local infrastructure between several Grids  Some solutions (under investigation): Information system: CrossView Fair share with different penalty factors Batch jobs worsen the priority according to the resources used Interactive jobs worsen the priority faster than batch ones Interactive CE Automatic injection of glidein wrappers for all jobs submitted from other sources different of CrossBroker

37. INGRID’08, Italy, April 2008 37 Int.EU.Grid added value  int.eu.grid offers Interactivity and Visualization “On the Fly” interaction, “On the Fly” response; Graphical VO applications Inter and intra cluster parallel tasks PACXMPI & OpenMPI User friendly access to resources The Migrating Desktop  int.eu.grid features go in favour of fulfilling most of the requirements of users from other major grids… VOs can take advantage of the interoperability solutions developed within int.eu.grid Add/configure VO resources without losing their normal (gLite) capabilities.