Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004

2 Outline The University of Bristol Grid (UoBGrid). The UoBGrid Resource Broker Users & their environment. Problems encountered. Other Condor use within Bristol. Summary.

3 The UoBGrid Planned for ~1000+ CPUs from 1.2 → 3.2GHz arranged in 7 clusters & 3+ Condor pools located in 4 different departments. Core services run on individual servers, e.g. Resource Broker & MDS.

4 The UoBGrid System Layout

5 The UoBGrid, now Currently 270 CPUs in 3 clusters and 1 Windows Condor pool. Central services run on 2 beige boxes in my office. Windows Condor pool only in single student open access area. Currently only two departments (Physics, Chemistry) fully engaged though more on their way. Remainder of large clusters still on legacy versions of operating systems, University wide upgrade programme started.

6 Middleware Virtual Data Toolkit. –Chosen for stability. –Platform independent installation method. –Widely used in other European production grid systems. Contains the standard Globus Toolkit version 2.4 with several enhancements. Also. –GSI enhanced OpenSSH. –myProxy Client & Server. Has a defined support structure.

7 Condor-G Resource Broker Uses the Condor-G matchmaking mechanism with Grid resources. Set to run immediately a job appears. Custom script for determination of resource status & priority. Integrated the Condor Resource description mechanism and Globus Monitoring and Discovery Service.

8 Resource Broker Operation

9 Information Passed into Resource ClassAd MyType = "Machine" TargetType = "Job" Name and gatekeeper URLs dependant on resource system name and installed scheduler as systems may easily have more than one jobmanager installed. Name = "grendel.chm.bris.ac.uk-pbs“ gatekeeper_url = "grendel.chm.bris.ac.uk/jobmanager-pbs" Make sure Globus universe, check number of nodes in cluster and set max number of matched jobs to a particular resource. Requirements = (CurMatches < 5) && (TARGET.JobUniverse == 9) WantAdRevaluate = True Time classad constructed. UpdateSequenceNumber = Currently hard coded in the ClassAd CurMatches = 0 System information retreived from Globus MDS information for head node only not worker OpSys = "LINUX“ Arch = "INTEL" Memory = 501 Installed software defined in resource broker file for each resource INTEL_COMPILER=True GCC3=True

10 Possible extensions to Resource Information Resource state information (LoadAvg, Claimed etc): –How is this defined for a cluster, maybe Condor-G could introduce new states of % full?? Number of CPUs and free diskspace: –How do you define this in a cluster? Is the number of CPUs set as per worker or overall the whole system? Same for disk space. Cluster performance (MIPS, KFlops): –This is not commonly worked out for small clusters so would need to be hardwired in but could be very useful for Ranking resources.

11 Results of condor_status

12 Load Management Only defines the raw numbers of jobs running, idle & held (for whatever reason). Has little measure of relative performance of nodes within grid, currently based on: –Head node processor type & memory. –MDS value of nodeCount for the jobmanager (this is not always the same as the real number of worker nodes. Executes currently only to a single queue on each resource.

13 What is currently running and how do I find out? Simple interface to condor_q Planning to use Condor Job Monitor when installed due to scalability issues.

14 Display of jobs currently running

15 Issues with Condor-G The following is a list of small issues we have: –How do you do some resource definitions for clusters? –When using condor_q –globus actual hostname job matched to is not displayed. –No job exit codes….. The job exit codes will become more important with increased number of users/problems. Once a job has been allocated to a remote cluster then rescheduling elsewhere is difficult.

16 The Users BaBar: –One resource is the Bristol BaBar farm so Monte-Carlo event production in parallel to UoBGrid usage. GENIE: –installing software onto each connected system by agreement with owners. LHCb: –Windows compiled Pythia event generation. Earth Sciences: –River simulation. Myself… –Undergraduate written charge distribution simulation code.

17 Usage Current record: –~10000 individual jobs in a week, –~2500 in one day.

18 Windows Condor through Globus Install Linux machine as Condor Master only. Configure this to flock to Windows Condor pool. Install Globus Gatekeeper. Edit jobmanager.pm file so that architecture for submitted jobs always WINNT51 (matches all the workers in the pool). Appears in Condor-G Resource list as WINNT51 Resource.

19 Windows Condor pools available through a Globus Interface from a flocked Linux pool Within three departments, currently there are three separate Windows Condor pools approximately 200 CPUs. Planning to allow all student teaching resources in as many departments as possible to have the software installed. This will allow a significant increase in university processing power with little cost increase. When department gives the OK then they will be added to the flocking list on single Linux submission system machine. Difficulty encountered with this setup is the lack of Microsoft Installer file. –Affects ability to use group policy method of software delivery and installation, directly affects how some computer officers view installing etc.

20 Evaluation of Condor again United Devices GridMP Computational Chemistry group have significant links with industrial partner who is currently using U.D. GridMP. Suggested to CC group they also use GridMP though after initial contact this was suggested to be very costly. e-Science group suggested that Condor would be a better system for them to use. Agreement from UD to do a published function & usage comparison between Condor & GridMP. Due to start this autumn.