1. Nicholas Coleman Computer Sciences Department University of Wisconsin-Madison ncoleman@cs.wisc.edu http://www.cs.wisc.edu/condor Distributed Policy Management and Comprehension with Classified Advertisements

2. www.cs.wisc.edu/condor A Distributed Policy Scenario › A user submits a job to Condor › The user has designed a policy defining requested services › Machines in condor pool have policies restricting the use of services › The user’s job won’t run - Why?  Is user’s policy to restrictive?  Was job rejected by machine policies?

3. www.cs.wisc.edu/condor Policy Management › Resource allocation challenges  Resource heterogeneity  Policy heterogeneity › How to allocate resources?  Conventional centralized allocation not sufficient  Solution: Matchmaking with Classified Advertisements (ClassAds)

4. www.cs.wisc.edu/condor Job ? [ MyType = “Job”;............. Rank =... Requirements =... ] [ MyType = “Job”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] [ MyType = “Machine”;............. Rank =... Requirements =... ] Matchmaking

5. www.cs.wisc.edu/condor Classified Advertisements › Represent entities (e.g. jobs, machines) and their policies › A ClassAd is a set of named expressions called attributes › Types of attributes:  Characteristics of an entity ( Arch, OpSys, Memory )  Constraints for requested resource ( Requirements )  Preferences for requested resource ( Rank )

6. www.cs.wisc.edu/condor Typical Classads [ Type = “Machine”; KeybrdIdle = ‘00:23:12’; Memory = 256M; LoadAvg = 0.042969; Kflops = 21893; Arch = “INTEL”; OpSys = “LINUX”; Name = “foo.cs.wisc.edu”; Rank = (DayTime() >= ‘9:00’) && ((DayTime() <= ‘17:00’) ? 1/other.ImageSize : 0); Requirements = (other.Type == “Job”) && (other.Owner != “riffraff”) && (LoadAvg < 0.3) && (KeybrdIdle > ‘00:15’); ] [ Type = “Job”; Owner = “ncoleman”; Cmd = “run_sim”; Memory = 31m; Rank = KFlops/1E3 + other.Memory/32; Requirements = (other.Type == “Machine”) && (other.Arch == “INTEL”) && (other.Opsys == “LINUX”) && (other.Memory >= 128); ]

7. www.cs.wisc.edu/condor Policy Comprehension › Why won’t my job run?  My policy is too restrictive  My job is rejected by machines in the pool › Looking for answers  Use condor tools (condor_q, condor_status)  Stare at job ClassAd to find out what’s wrong

8. www.cs.wisc.edu/condor Condor Tools › condor_q –analyze: Of 105 resource offers, 105 do not satisfy the request's constraints 64 resource offer constraints are not satisfied by this request › User wants more details:  Which parts of job requirements expression are problematic?  Is job ClassAd missing any attributes?

9. www.cs.wisc.edu/condor Two Cases to Examine 1. No machines meet the job’s requirements 2. The job does not meet any machine’s requirements One or both of these issues may be preventing the job from running, but they are not interdependent. We can analyze each one separately.

10. www.cs.wisc.edu/condor [ Requirements = (Arch==“SPARK”) &&(OpSys==“SOLARIS2.7”) ] JOB Example 1

11. www.cs.wisc.edu/condor [ Requirements = (Arch==“SPARK”) &&(OpSys==“SOLARIS2.7”) ] JOB Example 1 Result: (Arch == “SPARK"): did not match - suggestion: REMOVE (Opsys == "SOLARIS2.7"): matched: 2 - suggestion: KEEP

12. www.cs.wisc.edu/condor [ Requirements = (Arch==“ALPHA”) && (OpSys==“WINNT”) && (Memory>=64) ] JOB Example 2

13. www.cs.wisc.edu/condor [ Requirements = (Arch==“ALPHA”) && (OpSys==“WINNT”) && (Memory>=64) ] JOB Example 2 Result: (Arch == "ALPHA"): matched: 1 - suggestion: REMOVE (OpSys == "WINNT"): matched: 2 - suggestion: KEEP (Memory >= 64): matched: 4 - suggestion: KEEP

14. www.cs.wisc.edu/condor [ Owner = “jsmith”; ImageSize = 120000; Requirements =... ] [ Requirements = (ImageSize <= 50176) && (MemoryReq < 49) ] [ Requirements = (ImageSize <= 115712) && (MemoryReq < 98) ] [ Requirements = (ImageSize <= 50176) && (MemoryReq < 49) ] JOB MACHINES [ Requirements = (ImageSize <= 50176) && (MemoryReq < 49) ] [ Requirements = (ImageSize <= 115712) && (MemoryReq < 98) ] 1 2 3 4 5 Example 3

15. www.cs.wisc.edu/condor   115712 50176 ImageSize MemoryReq 1,2,3,4,5 3,5 4998 

16. www.cs.wisc.edu/condor [ Owner = “jsmith”; ImageSize = 120000; Requirements =... ] JOB Example 3 Results of Test --------------- The following attributes are missing from the job classad: MemoryReq The following attributes should be added or modified: ImageSize: - suggestion: use a value less than or equal to 50176 MemoryReq: - suggestion: use a value less than 49

17. www.cs.wisc.edu/condor Current Work › ClassAd analysis prototype implemented in Java  Job requirements analysis  Machine requirements analysis  Current version supports a simple menu driven interface › Working on integrating with Condor tools  condor_q –analyze  condor_status

18. www.cs.wisc.edu/condor Future Work › Applications to other uses of ClassAds in Condor › Analysis of a successful match › Graphical interface › Analysis of gang matching › ClassAds as an authorization language

19. www.cs.wisc.edu/condor Conclusions › Automated fine grained policy expression analysis is useful and feasible › Different issues arise with job requirements analysis and machine requirements analysis › The ClassAd language is ideal for these purposes.